How do oracle services work under the hood?

The simplest explanation of how oracles work would be to look at the Chainlink Runlog initiator.

1. An oracle contract is deployed identifying that it is an oracle service. This is an on-chain contract.
2. An off-chain service is deployed. This will not be apparent on the ETH chain, but it will be connected to an ETH wallet. This may be something like a VM or docker job that can make API calls.
3. This VM/off-chain infrastructure will "watch" the ETH chain for transactions that have data identifying using them. The "oracle" will be a combination of this on-chain contract, and this off-chain service.
4. Another smart contract creates a transaction that calls the specifying ID of this oracle, requesting off-chain data, for example, via an API.
5. Our oracle sees this transaction, and creates the API call off-chain. When the data is returned from the API call, the oracle creates a new transaction on-chain, with this data.
6. The smart contract that originally requested the data, now pulls data from this new transaction.

This is it in a very simple nutshell. You'll notice you need gas:

1. For transactions (ETH)
2. For oracle (LINK, or other crypto)

I'll take a stab at a very high-level overview by describing a generalized goal, the limitations of Smart Contracts, and the outlines of a solution. This will hopefully provide some context for the implementation details.

1) The goal. A Smart Contract that can react to an observable event or data point in the universe. With so much data sloshing about on the internet, it's natural to contemplate automated responses to things that happen. This is so common outside of blockchains, it can seem surprising that it's so difficult within blockchains.

2) The problem. A Smart Contract can't reach out for data like a typical web server could attach to an API or RSS feed (or screen scaper, etc.) At a fairly deep level, there's a real problem here, because everything on a blockchain has to be verifiable by all miners. Any such external sourcing of inputs would also be a source of uncertainty that would confuse verification. Everything in the chain has to be verifiable not just by one server, but by ALL servers, and not just today. Forever.

How would any node confirm, years from now, that today, at precisely this time, a certain URL produced a certain data stream? If it can't do that, then how would it confirm the truth of the data in the chain? If that's not reliable (it isn't), then things will fall apart. If the blockchain isn't objectively verifiable using only the data in the blockchain itself, it doesn't work.

3) The Oracle solution. We can imagine a trusted person pressed into service (poor guy). He has a privileged account that can inject data he receives from outside the blockchain, and he's depended on to update the blockchain at intervals. Suppose he never sleeps and he updates the ticker data every few minutes.

We can imagine a simplified function along these lines:

function recordTickerData(bytes8 symbol, uint high, uint low, uint close) 
  onlyOracle 
  returns(bool success) 
{
  // store the data
  // possibly react to the data
  return true;
}

Great. Only the Oracle is allowed to inject data this way, but once the data is injected, we can start to do things with it, which is what we want.

The miners will be able to verify the blockchain. They won't be able to confirm that the Oracle's inputs match anything in the outside universe, but they will be able to confirm that the Oracle's inputs are allowed, and any resulting transactions are legitimate.

We've got some loose ends to address. First, the Oracle will need to sleep, and this is clearly a job for something automated. Second, every time the Oracle executes this function, he'll have to pay gas for the transaction. That means we'll have to pay the Oracle first so it has some funding from which to pay for the gas.

See where this is going?

Without getting into implementation details, the Oracle will have to resolve the incoming data, from a source, and map it to the contract's interface. There will be details about how often to call the function, and so forth. And there will be the administrative issue of funding since the Oracle will need a continuous source of funding to pay the gas for its periodic data injections. Anyone with serious funding involved will want to verify the Oracle arrangement is trustworthy because its inputs will have non-trivial consequences. So we have a requirement of selecting a data source that will be widely agreeable. We have some broad brush-stroke outlines of generalized Oracle solutions - let's have a trustworthy external service that maps outside data to one of our input functions and periodically sends update transactions.

Opinion markets like Auger and Gnosis present a slightly different take on information from the outside world. Suppose your contract depends on a one-time true/false condition and it isn't obvious that there's an outside source you can know about, in advance, that will transmit a simple truth to your contract.

For example, will Trump win the 2017 election?

Opinion markets provide a way to set up the question and, essentially, outsource the determination of the truth to a human community. It takes the form of a betting market that ultimately settles the question in a way you can configure in advance.

In pseudo-code, your contract can consider:

if(AugurQuestion(123).isDecided and isTrue) then ... do stuff.

I hope the foregoing sets the stage, somewhat, and provides some context to the implementation details of these efforts.

Edmund Edgar from Reality Keys here.

Basically what needs to happen is that somebody says what data they want, the trusted service provides an ID that identifies it (in our case a hash of the data) and a key they'll sign the result with, and somebody sends a transaction to the contract. Then we wait for the result. Once the result is known, the trusted service signs it with their key, somebody sends another transaction containing the signed data to the contract, and the contract does whatever is supposed to happen.

This can be done either on-chain or off-chain. We do this off-chain, so we just provide signed data and a key, and users grab this data off our site and send it to their contracts themselves. This would normally be part of a DApp, so for example PredictionToken have a screen that creates a transaction to grab the ID from our site and sets up a contract, and another screen to grab the signed data from our site and send a transaction to settle the contract. Oraclize do it on-chain, so your contract sends a request to their contract, and their contract writes an event to the event log and returns an identifier. They have a process watching the event log, and when the data is ready they send it as a signed transaction to your contract, which checks the sender of the contract (this is another way of checking who signed the data) and does whatever it's supposed to do.

From a developer's point of view the advantage of doing it on-chain is that you only have to deal with requests to one place (your node's RPC interface) and if you know exactly when the response will be due, your user only has to send one transaction rather than one to set up the contract and another to send the result. The downside is that it's fiddly to test properly (you either have to mock up their service or use a live test network) and there are nasty edge cases involving gas usage. (There are some ways to do this right, but if you look at actual contracts using Oraclize, they would often allow Oraclize to drain all money from the contract balance.)

Note that since you can never completely trust the service (Oraclize have a cryptographic proof to support their claim that the data they fetched really came from the site they say, but the contract can't check it so it doesn't really help) you might want to have multiple services doing the same thing, and have the contract require multiple signatures. This is also something you might want to do in a permissioned blockchain context, where already you have multiple known participants running nodes and signing blocks, so it would make sense to get them to sign the data.

They have themselves a contract that interfaces with the real world.

1. you register with their contract and define a function they should call
2. their contract is read by an external entity, that sees your "query"
3. the external entity executes the query (eg. calling random.org)
4. calls your callback function either directly or through their contract
5. profit ;-)