Forums  > Trading  > Historical order book reconstruction API for crypto markets  
     
Page 1 of 1
Display using:  

tardis_thad


Total Posts: 5
Joined: Aug 2019
 
Posted: 2019-08-15 16:22
Hi, I'm a founder of tardis.dev which I've built to scratch my own itch as couldn't find anything on the web that would give me historical high resolution data for crypto markets (futures, swaps, options included) with affordable pricing for individuals hence created my own - hope some of you will find it as useful as I do.

It's raw non aggregated historical data API including full order book snapshots & incremental delta updates, tick by tick trades etc. Currently supported are BitMEX, Binance, Deribit, Bitfinex, Bitstamp, Coinbase Pro, Kraken, Crypto Facilities and OKEx.

There is a RunKit playground on the homepage so you can play around with the API and client libs that are available on GitHub (https://github.com/tardis-dev).

I'd really appreciate any feedback you have.

Thanks! Thadeus

nikol


Total Posts: 752
Joined: Jun 2005
 
Posted: 2019-08-15 17:16
It seems to me that:
- Latency for cross-exchanges arb business is important, therefore, for each of this exchanges you should measure and disclose it. And if you collect data from one single location, then its value drops.
- Cryptotraders store their own data (once you have your own feed and internal reconstruction of order book, the rest is just organizational). Once you have it, to start listening and storing new exchange is the matter of 1 day or 2.
- Consistency: for example, by construction, my data are internally consistent with my API from the very beginning.

On the positive side: Web-site design looks good.

I really do not want to discourage. If you believe in it, shoot it.
Good luck!

tardis_thad


Total Posts: 5
Joined: Aug 2019
 
Posted: 2019-08-15 17:35
Thanks, really appreciate the feedback, not discouraged at all, it's a learning experience for me as I'm long time programmer not algo trader. Noted about latency. I measure it already, but it's not yet available via API and not sure if measurements are granular enough, will think about it. I'm aware about single location cons, but right now can't afford to have it colocated per exchange and in some cases 'middle location' (London) can be good enough for certain scenarios (eg: arb between BitMEX - Ireland and Deribit-France) at least it was my line of thinking. I'm aware that more professional firms often invest in own infra that collects the data, but I wouldn't call it 1 or 2 day job - at least it was mine experience. Not sure what you meant by consistency, but data provided by the API is in exchange format so I assume that anyone trading on the exchange will have to know the format anyway and it will be easy enough to convert it to desired format anyway as it's what needs to be done for real-time data anyway.

I wanted to reach people in similar situation to mine - "indie algo traders", but perhaps your're right it's non existing market for this kind of data as most of such traders don't use tick data or store it on their own and larger players have resources to build own infrastructure where they control everything. Anyway this API is something I'm using so at least it's that :)

nikol


Total Posts: 752
Joined: Jun 2005
 
Posted: 2019-08-15 18:42
> API is in exchange format

Yes, of course, you talk about json. Confirmed.

About latency: you can record your PC clock to all data ticks, so everyone can see it while streaming data. If data is collected over the grid of servers, then find the way to sync all those clocks. In principle, most of exchanges have minimal latency of x10 ms (except very few). It is huge comparing to the scale of under-mks, but still...

tardis_thad


Total Posts: 5
Joined: Aug 2019
 
Posted: 2019-08-15 19:07
Indeed, most of the crypto exchanges real-time streams are WebSocket and JSON based, I know some provide FIX but it's not that popular in this space and even it there is FIX it's often built on top of WebSocket (eg Bitfinex).

I see now what you mean about latency, I already do that and each message received has local timestamp stored as well as it's all available via API. I also measure every minute ping and http request/response time for each exchange and assumed it's what you were talking about.
When researching similar services in crypto space I couldn't find historical order book data that was tick level, what I could find was at most 1s snapshots with really high price tag and that pushed me to build something like this - essentially crypto market data replay API - in principle similar to https://www.dxfeed.com/market-replay/ but of course not the same scale.

nikol


Total Posts: 752
Joined: Jun 2005
 
Posted: 2019-08-15 19:11
see this discussion

http://www.nuclearphynance.com/Show%20Post.aspx?PostIDKey=184987

tardis_thad


Total Posts: 5
Joined: Aug 2019
 
Posted: 2019-08-15 19:51
Thanks, I'm already familiar with CoinAPI, it's a fine service and some good points in that thread, although I'm pretty sure they do not provide non aggregated historical order book data and it's only 20 top bids/asks if I understand correctly their docs hence not something I was after as wanted to have a way to reconstruct full market state in any given moment in the past tick by tick.

nikol


Total Posts: 752
Joined: Jun 2005
 
Posted: 2019-08-16 17:23
ICE is your competitor?

"The feed delivers real-time and historical data for the most widely and actively traded cryptocurrencies. Covering 60+ cryptocurrencies from leading venues, markets and exchanges across the globe."

https://www.theice.com/market-data/connectivity-and-feeds/consolidated-feed/coverage-list/cryptocurrencies

tardis_thad


Total Posts: 5
Joined: Aug 2019
 
Posted: 2019-08-16 18:09
Yes, it's in the same market space. It has partnered with blockstream https://blockstream.com/cryptofeed/ to provide such data. There isn't much details what that data feed includes exactly, but I'm pretty sure it's not historical order book data, but real-time only. If anyone has access to it I'd love to know! Overall it feels a lot enterpisey. BTW, I'm not sure why one would like to connect to exchange via 'proxy' that is additional potential point of failure, when can do this directly for free, but perhaps it works fine for certain use cases.

nikol


Total Posts: 752
Joined: Jun 2005
 
Posted: 2019-08-16 20:40
Cannot help, sorry. For me it was a discovery as well, related to news about Bakkt 'cleared to launch Bitcoin futures' via ICE for institutional investors with physical settlement (i.e. with Bitcoins). No idea where institutionals will get BTC if they are officially not yet in the game.
Previous Thread :: Next Thread 
Page 1 of 1