Extracting data from the tangle

Basically there are three ways to get to the data (depending on what you want):

• Query the API of your node to find transactions and walk along them. This might be slow (depending on what you are looking for), but the API is well-documented and you can walk the tangle quite easily.

• Directly query the node's database (which is a RocksDB instance) for what you need. This is probably the fastest way (from CPU time perspective) but can be frustrating as the database structure is not documented and there are a few quirks here and there which are usually handled by iri but you'd have to handle them yourselves this way.

• Write an IRI Extension module (IXI) that collects the data you need and aggregates it. This is mainly useful if you want to aggrenate data in a way that the full node already has code for (e. g. Snapshot.ixi, which is used as base for address distribution statistics)

All these ways will give you all kind of information about the tangle, transactions stored therein and address balances. You won't get information about full nodes or number of "real users" that way, though.

Also, the information stored in the tangle will stop at the last snapshot (which was in November). As there are no public permanodes so far, the only way to get older transaction data is to download old IRI databases, import them into the corresponding iri version and re-run your gathering task against those nodes (in the hope that the API did not change too much in between).