your points are well taken but unfortunately i donāt think that designing algorithmic policy in the crypto space is anywhere near mature enough to be focused primarily on comprehensible, comprehensive and consistent. These are better thought of as properties of the results documentation around a particular model or design. To quote Sebnem R āversion control your assumptionsā.
For a model Comprehensive is unachievable in practice. You cannot possibly cover all related concepts in a single model. Instead we rely on the model builders discretion in what is and is not out of scope in any particular case. In fact this means that we often require multiple models in order to provide different perspectives on the same system design.
this leads us to consistent. There are lots of ways to interpret this term but taking your explanation at face value, I strongly agree that clearly defining conserved quantities is important. This means stock and flow modeling in the system dynamics sense is very important for tracking conserved quantities even when the model is itself agent based. However, returning to the comprehensive it is often necessary to use subpopulation models, mean field models or other techniques to ensure models are not exploding in computational complexity, this can also help with comprehensibility at the cost of being reductions of some aspect of the system. Furthermore, models at different scales or scopes may actually be inconsistent with one another. Rather than interpret either model as wrong, the divergences need to be inspected for insights into the system at hand, and inform further models or experiments.
this leads us to comprehensible, models should be as simple as possible but no simpler. In a sense this means they are maximally comprehensible but how simple this actually is depends a lot on what your models purpose is. In the case of safety critical system it may actually still be quite complex, with multiple safeguards in place to protect the system from achieving states where its properties would break down. Another way to look at it is, a model may actually be a model of models each of which is comprehensible but the overall system is a lot to take in for the non-expert. (See model based systems engineering).
I will grant you that I am talking about formally engineered algorithmic policies like those you find in aerospace, defense and cyberphysical systems. If one is talking about a simple stand alone business process without a concern systemic interactions then some basic business logic will suffice and certainly anything making it opaque to the user will only harm its usefulness. It hard to achieve even approximately ācomprehensiveā with that reduction of scope though.
The trade-offs in meeting your stated goals aside, the purpose of this thread is to help people get their hands dirty and their heads around modeling systems which have non-trivial interaction effects through simulating those system models.
I would argue comprehensibility takes a different form:
- making explicit the properties of the model rather than the details of why it has those properties (this includes mathematical claims, and their proofs when appropriate)
- this also include numerical demonstrations of those properties
- this includes plaintext descriptions of why those properties matter to users and/or policymakers
- also any material provided should be not only comprehensible but also verifiable by other experts.
In part the goal is to organically expand the set of such experts to the point where this is a viable approach. I would argue that today, there are too few people with formal training in automation related fields within the broader crypto space, and even fewer with both training on automation and social sciences required to contextual algorithmic policies is social systems.
Iāll grant you that this quest wonāt naturally teleport the quest doer into a professional engineer with experience in design, testing and maintenance of algorithmic polices, but it is an attempt to give people a chance to learn parts of that field without going back to university for a degree in robotics and control or automation. The work linked is imperfect and incomplete. There needs to be much much more.
So interpret this quest as a part of an ongoing effort to identify resources that will help people learn, explore new ideas and eventually teach others to do the same. After all modeling social systems is very hard (not necessarily making a model but understanding the implications of what you have constructed), whatever assumptions one makes (implicitly or explicitly) get baked into any conclusions one arrives at in the end. This is doubly important to remember when using these models to design algorithmic policies.
Itās a long road ahead of us, this quest is just an invitation to join the party.
