Must create now a similar history using poloniex. We need to reconstruct it for each currency:
Creating CSV-files with ';' as separator. Added an option --dump for poloniex, which creates CSV files. Corrected content of active_loans.csv. Small refactoring of lib/rapflag/poloniex to separate loading all information into a private method load_history_info and storing it into various read only variables, with names like trade_history.
Fixed a small error I introduced yesterday in the poloniex gem with Fix typo in default parameter for lending_history
Pushed commit Added -dump option and corrected poloniex CSV files.
Now we are able to generate a dump of the history related data in poloniex into the subdirectory output via calling bundle exec bin/poloniex --poloniex --dump or bundle exec bin/poloniex -p -d
Adding loading and dumping complete_balances, which are needed for the analysis. Done with commit Added complete_balances
For unit tests and developing the extract algorithm I add a utility procedure to read in existing CSV files. Done. Observed, that the lending history is not sorted by date. Should we sort it before saving it to a CSV file? Yes. But we do it with a helper function, which can be called at any time.
Correcting deposits_withdrawls -> deposits_withdrawals in rapflag (but not the spelling error in poloniex). Also withdrawals and deposits have different fiels in @deposits_withdrawals as seen by pry
[14] pry(#<RAPFLAG::Poloniex>)> @deposits_withdrawals['withdrawals'].last
=> {"withdrawalNumber"=>3053682,
"currency"=>"BTC",
"address"=>"16uwdWzEBQAysWdGJbrkCkACgsDYYPe99a",
"amount"=>"8.06684595",
"timestamp"=>1491166881,
"status"=>"COMPLETE: 06fa7363b96cfcb63a13094b5e0ec5a624821853a9bde3213a599c1040346f41",
"ipAddress"=>"212.35.14.109"}
[15] pry(#<RAPFLAG::Poloniex>)> @deposits_withdrawals['deposits'].last
=> {"currency"=>"BTC",
"address"=>"12jQat5WU97uPb8Ky5nrKAeJJAJPzN3e7g",
"amount"=>"15.70418342",
"confirmations"=>1,
"txid"=>"fe0ee205f51c93794cb896a77edc368dd17312082e3a100919f48d1ecf719cbc",
"timestamp"=>1487845873,
"status"=>"COMPLETE"}
I think the best thing is to split this list after reading it via the Poloniex API into two array deposits and withdrawals.
Asking myself whether it would be worth to convert all the hashes into openstructs, .eg array=[];@active_loans['provided'].each {|x| array << OpenStruct.new(x) };. Yes I think it is better for create some new arrays for each loan type, currency_pair, etc. Also a OpenStruct can be easily converted into an Array to dump it, eg. a_ostruct.to_h.values and we can generate automatically the correct names for the header files using ostruct.to_h.keys.
Pushed commit Save/load results via spec/data/*.json
Adding procedure to:
Some sums look right for the daily summaries, but other entries are missing. Pushed commit First try at calculating daily summary
It is unclear for me how I have to handle the trade_history, where I have keys like ["BTC_GNT", "BTC_XMR", "BTC_ZEC", "BTC_DAO", "BTC_ETH"], but only one account for 'GNT'.
I think that I must somehow correlate the address in withdrawals to the other used currency like
[5] pry(#<RAPFLAG::Poloniex>)> @deposit_addresses.find{|k,v| v=@withdrawals.last.address}
=> ["BTC", "12jQat5WU97uPb8Ky5nrKAeJJAJPzN3e7g"]
[6] pry(#<RAPFLAG::Poloniex>)> @deposit_addresses.find{|k,v| v=@withdrawals.last.address}.first
=> "BTC"
[7] pry(#<RAPFLAG::Poloniex>)> @withdrawals.last
=> #<OpenStruct withdrawalNumber=3053682, currency="BTC", address="16uwdWzEBQAysWdGJbrkCkACgsDYYPe99a", amount="8.06684595", timestamp=1491166881, status="COMPLETE: 06fa7363b96cfcb63a13094b5e0ec5a624821853a9bde3213a599c1040346f41", ipAddress="212.35.14.109">
@deposits.collect{|x| x.address}.uniq
=> ["12jQat5WU97uPb8Ky5nrKAeJJAJPzN3e7g", "0x564cf247553bd367f31cf830eaec80ff6e2b94b9"]
[16] pry(#<RAPFLAG::Poloniex>)> @withdrawals.collect{|x| x.address}.uniq
=> ["3BMEXRVMUzkSTKt7LoAEgsfBUS58xdVBhR",
"0xbb9bc244d798123fde783fcc1c72d3bb8c189413",
"3B3RKNccK3Q1LRod8Je6YXhtSwFbDiXAE1",
"1MPPvcwEgYgMMZwQwvULQfZGXCc69ZQSvn",
"3Dbhj3gSui9EZgXWHPQUkEE9Dw797nAhQX",
"3FB1atX6whqteKopKaUCugtKjZwSct5z5L",
"15V2EJBp2tkoFkgAB6SPJTRfDFbaUHJ9AW",
"16uwdWzEBQAysWdGJbrkCkACgsDYYPe99a",
"t1e2M1RfKt3K9FLKRx6XKkkzmCf3igoUZ6i",
"4BCeEPhodgPMbPWFN1dPwhWXdRX8q4mhhdZdA1dtSMLTLCEYvAj9QXjXAfF7CugEbmfBhgkqHbdgK9b2wKA6nqRZQCgvCDm"]
Pushed commit Fixed erroneous writing of json files in spec/data