How to cadCAD 📈

This Quest Tree will begin with some Introductory Materials on Modeling and Simulating Systems. It will grow (and change over time) based on feedback from metagame players!

An up to date outline of the quest tree will be maintained in this top level post along with estimated seeds (use your best judgement, effort varies widely by past experience level).

  • Learn the basic Modeling Concepts (~1 hour)
  • Install cadCAD and run a demo model from the list provided (~1 hour)
  • Work through the Robot and Marbles Tutorial Series (~half day)
  • Revisit basic Concepts and read advanced concepts (~2 hours)
  • Pull a more advanced (working draft) model and modify it (~half day)
  • Create a New Quest (or Questline) in this Quest tree (~1 hour)

I expect this to provide Modeling and Simulation XP in the future :wink:

11 Likes

M&S Quest 1: Learn the basic Modeling Concepts

This quest is design to buff up your background knowledge for modeling and simulation in preparation for the rest of the modeling and simulation questline

Reading List:

To complete this Quest

  • author a new post within the community.cadcad.org discourse which either asks a new question about simulation and modeling or answers a prior question
  • tweet about how you are learning to model/simulate the metagame
  • post a link to your forum post and/or your tweet in discord
5 Likes

M&S Quest 2: Install cadCAD and run a demo model

This quest is designed to provide you with new equipment to aid you on your quest!

If you are new to python in addition to cadCAD start here:

If you wish to bypass Python setup for now, you can use google colab

If you are a python user and want to get down to business then open your terminal:

  • pip install cadCAD

Now that you can run cadCAD let’s try it out. There are a variety of sample models available.

  • A model of the ETH2 beacon chain by the Rigorous Incentives Group (RIG) at the Ethereum Foundation [Github]
  • A model of the ETH-DAI pair on Uniswap created by Markus at BlockScience [Github]
  • A model of the verifiers dilemma [Github] [Medium]

There are other models you can try out but these are relatively well documented so they are good starter models to play with.

To complete this Quest

  • write a tweet with a link to github of the model you ran and a screenshot of YOUR results
  • make sure to share what you learned by from your experience!
  • post a link to your tweet in the discord
4 Likes

M&S Quest 3: cadCAD Tutorial Series

This quest is designed to teach you how to use your newly acquired equipment. The robot and marbles tutorial series is interactive feature documentation demonstrating the use of the cadCAD software for a simple system.

Robot and Marbles Tutorial Series

In this series, we introduce basic concepts of cadCAD and system modeling in general using a simple toy model. [Github]

Part 1 - States and State Update Functions
Part 2 - Actions and State Dependent Policies
Part 3 - From Synchronous to Asynchronous Time
Part 4 - Uncertainty and Stochastic Processes
Part 5 - Using class objects as state variables
Part 6 - A/B testing

Check out the videos folder for detailed walkthroughs of each one of the tutorials.

For details regarding specifying model configurations and simulation configurations you may wish to refer to the documentation.

Warning: It will take you half a day to complete this quest properly and you will probably wish to spread that work out over a week

To Complete this quest

  • Go to community.cadcad.org and write a post giving feedback on the tutorials; include what you found helpful as well as what could make the tutorials easier to learn from. You may also wish to recommend a future addition to the tutorial series.
  • tweet about accomplishing and highlight what you learned
  • post a link to your forum post and/or your tweet in discord
4 Likes

M&S Quest 4: Advanced Modeling Concepts

Now that you understand how the cadCAD software works, it’s time to start learning to model. As you learned in the introductory modeling quest, modeling is a subjective representation of a phenomena. Mathematics is the language used to represent these phenomena but it’s also important to build intuition as well as to be clear about your goals. This quest provides additional reading material and references to help build those skills, preparing you to strike out on your own modeling journey.

Broad perspectives on modeling to help guide your understanding of complex systems.
Foundations of Cryptoeconomic Systems (MIT Cryptoeconomic Systems Conference 2020)

Motivating Systems Engineering
Synthesis —The Essence of Systems Engineering (Derek Hitchen’s self published)

Curated Reference materials on wide rang of mathematical topics
Token Engineering Learning Program (Blog by Tannr Allard)

Core Reference for Economics of Dynamic Stochastic Processes
Ergodicity Economics (Extensive Lecture Notes from London Mathematical Lab)

Definition of Discrete Event Games and Formal Application to Bonding Curve Price Dynamics
Economic Games as Estimators (Vienna University of Economics, Working Paper Series)

The above references would take far more than 2 hours to read exhaustively and understand deeply. The purpose of this quest is familiarize the quester with additional references and to invite reflection on the basic concepts in light of learnings from quests 2 and 3.

To complete this quest

  • Choose one of the readings (or subsections of the longer references) and write a post summarizing what you learned from reading it and how you plan to apply that learning on your future modeling journey.
  • tweet about accomplishing this quest and highlight what you learned
  • post a link to your forum post and/or your tweet in discord
4 Likes

M&S Quest 5: Forking to Build Your own MetaGame Model

Having reviewed the advanced concepts it’s time to start doing your own modeling. Don’t worry you don’t need to start from scratch. In this quest you will start with a relevant existing model relating to a metagame concept and start modifying it with your own ideas.

Recommended Base Models to Fork

Three-Sided Market [Github]

The three-sided market archetype is characterized by ‘Community-Corporation Substitutability’. That is to say that a community coordinating through technology such as decentralized ledgers can functionally fulfill a role in the economy that is currently fulfilled by a corporation without the users of that corporate product knowing or caring that the provider is a community. This model is focused on making via platform businesses, no business required.

Cyber-Physical Commons [Github]

Using this infrastructure, community-focused organizations can be spun up that are continuously funded, self-governed, and live within their own cryptoeconomies. They have their own community token, with guaranteed liquidity to solve the convertibility problem of local currencies. Their communities are held socially accountable, duly represented, and financially sustained. [Source]

SourceCred on a Dynamic Attribution Network [Github]

SourceCred as defined essentially gives the maintainer the powers of a game designer. However, anyone who has worked in the gaming industry knows that balancing games is quite hard. There is no one magical setting such that everything always works out forever. In an effort to empower open source communities, we are also creating the potential destroy the ecology that has made them so successful. It is important to also provide the guidance, and where possible tools to help maintainers be good stewards of their communities. [Source]

To Complete this Quest

  • Come up with a novel question related to the MetaGame: model, simulate and interpret the results
  • Write a post explaining the question, how and why you approached modeling it and link to your simulation code; highlight any new questions that arose as a result of this research
  • tweet about accomplishing and highlight what you learned
  • post a link to your forum post and/or your tweet in discord
4 Likes

M&S Quest 6: Party up for Open World Questing!

You have completed the newbie zone and are ready to venture out on your own. Add Quests in the M&S Quest tree to encourage other members of the metafam to explore the metagame with you!

To Complete this quest

  • Write a Quest post that poses a specific question relevant to the metagame in enough detail to allow others to discuss it and participate in modeling, simulating or analyzing data generated by simulations
  • If you have already started building a model share a github link
  • Be specific about what you are working on and how others can contribute to the quest but keep it open ended; be sure to push the limits, simulations are computer aided thought experiments!
  • share on twitter and in discord to recruit more party members
4 Likes

@mzargham This is so cool! Thank you for putting all of this together :slight_smile:

Also, I think I found our first thing to model! lol

https://forum.metagame.wtf/t/mg-quest-2/43/2

2 Likes

@mzargham if we have questions or comments related to this quest, does it make sense to drop them into this thread (could get messy), or should we create a new category for each Quest where then there can be an About thread, Q&A thread, and whatever else (could get overwhelming)?

For example I’m drawing out a map of the MetaGame dynamics and the players, and it would be great to crowdsource a list of all the things to track so that we don’t miss anything

2 Likes

M&S Quest 1 Complete!

https://community.cadcad.org/t/on-modeling-agents-in-dynamic-systems/23/9

Note this ended up taking me ~ 3hrs, not the 1hrs that was advertised, but… totally worth it! :slight_smile:

2 Likes

M&S Quest 2 Complete!

https://twitter.com/CheeseYet/status/1236787565425876992

This is my first time running CadCAD. Went with the ETH2 beacon chain by the Rigorous Incentives Group (RIG) at the Ethereum Foundation [Github]. From what I can tell, the idea is to define a system in python with actions, rewards, and punishments. Then you can use pass those actions into CadCAD and model how agents might fair taking said actions in repeated games. It’s kind of like machine learning, but you’re discovering how a system will evolve based on a time horizon, players available actions, and their policy for choosing said actions. Maybe we should call it “machine discovering?”

Also, would be very very cool if we could set things up such that the machine learns a model that optimizes parameters (rewards/punishments) to find an equilibrium. My guess is that this is already a thing, but probably computationally expensive as you’d have to model agents as RL algorithms and then a meta-model on top to optimize the rules of the game based on the actions of said agents. Kind of like auto-ml, but for tokeneconomic design. @mzargham is this something that BlockScience already does and/or are there better ways to think through tokeneconomic systems than just throwing massive amounts of compute at the problem? lol

3 Likes

Given the way the thread is set up and the expansive nature of cryptoeconomics questions, I think we should move them into their own thread. If the questions are more cadCAD methods and tools specific, the conversation should move to community.cadcad.org so that we do not need to duplicate discussions across forums.

Sorry about not being super responsive here. I had a lecture at MIT yesterday, an academic paper deadline today and I have a talk in San Fransisco tomorrow. Things are going to continue to be hectic for a week or so.

Another upside of taking cadCAD questions to the cadCAD forums is that other people there can also answer your questions.

1 Like

Yeah but we kind of need to engage here to get that all that sweet sweet MetaGame XP

@burrrata, Thanks to AraCred!

Discussions of modeling in community.cadcad.org will generate cred in the TokenEngineering Commons, which in turn, I hope to see earning cred in the metagame by contributing to the project. That case for metacred will be integral in realizing the positive sum metagame:
https://forum.metagame.wtf/t/realizing-a-positive-sum-metagame/33

For cred purposes @METADREAMER regarding metacred. That just seems appropriate doesn’t it.

4 Likes

100%. Ceramic network has my gears turning with ways to enable cross-community cred earning.

Essentially, decouple the “capturing of data and information for what happens” from the specific instance of SourceCred, as well as rest of the steps described here:

The information about “what happened” doesn’t belong to a specific community, it’s just a record of an occurrence. There are lots of occurrences that create value in multiple communities, but each community would have to redundantly capture this information in order to attribute value to it. If this information is made open and interoperable between communities, then multiple communities can attribute value to the same event/contribution/outcome. All while retaining their own autonomy and control over how their cred is distributed (i.e. the cred weighting / configuration of one community doesn’t affect another).

Here’s a diagram cuz diagrams are cool:

5 Likes

In latest AraCred work-session @mzargham and I explored how to work with multiple sources of value metrics - and as part of that we sketched out a mechanism that was very similar to the Ceramic documentation-chain concept. Seems like Ceramic would be a natural fit if/when it’s ready.

1 Like

I wrote a sort of MetaQuest for modeling quests here:
https://community.cadcad.org/t/quests-for-quests-aka-more-demo-models/145

If you finish the quest tree above and you are looking into diving into your own modeling

OR

if you are looking for someone who has some modeling experience and you want to provide them a quest to do token engineering work

this post will help you. Also feel free to join the token engineering discord where we’re focused on increasing the capacity for this kind of work by applying these skills and sharing experience:

3 Likes

So this is a quest which will build foundations for the blockchain Wizard?

I’ve done some system modelling b4 (in fact completed blockchain design) so one of the weaknesses is that models should be (in no particular order)

  1. comprehensible - broken down the key human (and perhaps org) stakeholders
  2. comprehensive - includes also bad actors, what incentives do they have in breaking
  3. consistent - the stocks/flows need to have external validity - even if utility token from bonding curve

My suggestions … see whether service to run models from service … much like node-red, it might be possible to implement as flows.
Factor in cost for optional coach … if competent won’t need but some will need assistance.
Based on burrata taking 3x longer you might have to recalibrate actual hours expected (triangle distribution?)

your points are well taken but unfortunately i don’t think that designing algorithmic policy in the crypto space is anywhere near mature enough to be focused primarily on comprehensible, comprehensive and consistent. These are better thought of as properties of the results documentation around a particular model or design. To quote Sebnem R “version control your assumptions”.

For a model Comprehensive is unachievable in practice. You cannot possibly cover all related concepts in a single model. Instead we rely on the model builders discretion in what is and is not out of scope in any particular case. In fact this means that we often require multiple models in order to provide different perspectives on the same system design.

this leads us to consistent. There are lots of ways to interpret this term but taking your explanation at face value, I strongly agree that clearly defining conserved quantities is important. This means stock and flow modeling in the system dynamics sense is very important for tracking conserved quantities even when the model is itself agent based. However, returning to the comprehensive it is often necessary to use subpopulation models, mean field models or other techniques to ensure models are not exploding in computational complexity, this can also help with comprehensibility at the cost of being reductions of some aspect of the system. Furthermore, models at different scales or scopes may actually be inconsistent with one another. Rather than interpret either model as wrong, the divergences need to be inspected for insights into the system at hand, and inform further models or experiments.

this leads us to comprehensible, models should be as simple as possible but no simpler. In a sense this means they are maximally comprehensible but how simple this actually is depends a lot on what your models purpose is. In the case of safety critical system it may actually still be quite complex, with multiple safeguards in place to protect the system from achieving states where its properties would break down. Another way to look at it is, a model may actually be a model of models each of which is comprehensible but the overall system is a lot to take in for the non-expert. (See model based systems engineering).

I will grant you that I am talking about formally engineered algorithmic policies like those you find in aerospace, defense and cyberphysical systems. If one is talking about a simple stand alone business process without a concern systemic interactions then some basic business logic will suffice and certainly anything making it opaque to the user will only harm its usefulness. It hard to achieve even approximately “comprehensive” with that reduction of scope though.

The trade-offs in meeting your stated goals aside, the purpose of this thread is to help people get their hands dirty and their heads around modeling systems which have non-trivial interaction effects through simulating those system models.

I would argue comprehensibility takes a different form:

  • making explicit the properties of the model rather than the details of why it has those properties (this includes mathematical claims, and their proofs when appropriate)
  • this also include numerical demonstrations of those properties
  • this includes plaintext descriptions of why those properties matter to users and/or policymakers
  • also any material provided should be not only comprehensible but also verifiable by other experts.

In part the goal is to organically expand the set of such experts to the point where this is a viable approach. I would argue that today, there are too few people with formal training in automation related fields within the broader crypto space, and even fewer with both training on automation and social sciences required to contextual algorithmic policies is social systems.

I’ll grant you that this quest won’t naturally teleport the quest doer into a professional engineer with experience in design, testing and maintenance of algorithmic polices, but it is an attempt to give people a chance to learn parts of that field without going back to university for a degree in robotics and control or automation. The work linked is imperfect and incomplete. There needs to be much much more.

So interpret this quest as a part of an ongoing effort to identify resources that will help people learn, explore new ideas and eventually teach others to do the same. After all modeling social systems is very hard (not necessarily making a model but understanding the implications of what you have constructed), whatever assumptions one makes (implicitly or explicitly) get baked into any conclusions one arrives at in the end. This is doubly important to remember when using these models to design algorithmic policies.

It’s a long road ahead of us, this quest is just an invitation to join the party.

2 Likes

Are there specific systems we’re looking to model here, or is this more of an open ended means of getting people up to speed on systems engineering tooling? I’m very interested in helping out here, but it would help to have some more context of the terrain.

1 Like