Zeno found an example, that the Preis Vergleich still showed outdated packages. The search by Preisvergleich did not list any outdated packages.
But selecting a single product and comparing, e.g. https://ch.oddb.org/de/gcc/compare/ean13/7680660970016 still list https://ch.oddb.org/de/gcc/compare/ean13/7680517950680 (IKNSR 51795 Package 068), which is no longer in Packungen.xlsx.
Verified with bin/admin that the expiration_date/expired? is correctly set
ch.oddb> registration('51795').package('068').expiration_date
-> 2017-05-29
ch.oddb> registration('51795').package('068').expired?
-> true
Therfore I must fix it.
Also in Einstellungen the text for Nur aktuell zugelassene Medikamente is not correctly indented. Fixed with commit Fix identention of 'Nur aktuell zugelassene Medikamente'
To limit the packages shown in the "Preisvergleich" I can use the comparables in src/state/drugs/compare.rb or src/model/package.rb. Which one should I choose.
Resolved the problem with outdate package in Preisvergleich with commit Limit comparable packages to non expired ones
We remarked yesterday the following two deficiencies:
The first point was a decision by Hannes Wyss in May 2008 in its commit https://github.com/ngiger/oddb.org/commit/571b16437#diff-f65539e500051180052a1afbec481498R468
In order to minimize the number of rows to compare and to have a nice statistic the swissmedic plugin uses our swissmedic-diff gem.
I think it would not cost so much more time and would be more robust just to compare completely the database entries versus the package.xlsx file. And this should bring the complexity down, as the file is just too large (over 1500 lines). It was less than 500 lines in 2008. And it was less than 900 when I took it over in 2012.
Instead of reporting changes between two XLSX files we would report the changes made to the database, which should ideally be the same. We could for some period continue to show the changes reported by the swissmedic-diff. And/or attach details of the swissmedic-diff reported changes and the changes we applied to the database.
When looking at the file and its git log history, I can resume its features (which will be part of the file after the next commit) like follows
Currently we have the options :check, :fix_galenic_form, :update_compositions, which
Must these option still be offered, or should we default to true for all of them if the corresponding fields have changed. Reparsing all compositions takes a long time (around 15) and therefore should probably be avoided.
Starting with creating a human readable data/xls/Packungen-2017.09.20.swisssmedic_diff.
I think that the methods initialize_export_registrations, set_all_export_flag_false, update_export_sequences and update_export_registrations should be grouped together at the end of update and maby even grouped into a separate method to be easily de/activated.
As reading the XLSX files takes minutes, we could use the ssconvert utitily from the Gnumeric application to convert it into a CSV files (takes about 2 seconds) and read in the CSV (which also will take less than a second). Afterwards a diff is easy for CSV files, too.
Alternatively we could also use csvkit from https://csvkit.readthedocs.io/en/1.0.2/index.html. I installed it on oddb-ci2 using sudo pip install csvkit. It took a little bit less than 10 seconds to run in2csv Packungen-2017.09.20.xlsx > Packungen-2017.09.20-1.csv. Comparing the generated csv file I see that csvkit formats date fiels like 2017-08-31 whereas ssconvert produces 2017/08/31.
Zeno has good arguments for the following decision
This should be fixed with commit Fix saving search_type
Must also fix the error that eg. limiting to SL entries only does not work with drugshortage. Reason: In src/state/drugs/result.rb the package_filter does not get applied.
Found a solution, but it takes a very long time to load the drugshortage list (about 140) seconds. But it can be limited now correctly (including the correct number of packages), eg. to only SL and group A. Must have a next week a closer look at whether my patch Attach:0001-Fix-search-preferences-with-drug_shortage.txt is correct or not.
Removed the unless pack.expired? after line 53 PackageFacade.new(pack, package) in src/state/drugs/compare.rb. Now without "Nur aktuell zugelassene Packungen" ti displays for a trademark show for Similasan 112 entries.After limiting only 82 are shown. Pushed commit Do not limit compare to non expired packages
Drugs with prices, eg.g
ch.oddb> registration('60486').package('002').price_public.class
-> NilClass
ch.oddb> registration('60486').sequence('01').package('002').name_base
-> Omeprazol MUT Sandoz 10 mg
ch.oddb> registration('65990').package('003').name_base
-> Omeprazol Zentiva 10mg
ch.oddb> registration('65990').package('003').price_public.class
-> NilClass
ch.oddb>
ch.oddb> registration('60493').package('046').price_public.class
-> NilClass
ch.oddb> registration('60493').package('046').name
-> Stadaprazol 10 mg, magensaftresistente Kapseln
The search https://ch.oddb.org/de/gcc/search/zone/drugs/search_query/Omeprazol%20MUT%20Sandoz/search_type/st_sequence?#best_result one finds only prices for the packages 006, 007 and 008, but not for the package 002