Index>

20160817-restart-migel-after-importBauerfeind

Summary

• After importing Bauerfeind products we must restart the migel daemon
• brand2csv must eliminate all duplicates
• evidentia search links not correctly updated
• Keep in Mind

Commits

• https://github.com/ngiger/migel/commit/1cdfc675ab7acb1afc5454361b1317d690b68462 Restart migel server after bauerfeind import
• https://github.com/ngiger/migel/commit/2590f26d9658bb9ba5929f07e7ac5340d984d966 Use awk to strip leading spaces from ps
• https://github.com/ngiger/oddb.org/commit/a9f771f37f0415c16410ccc590d127975679e043 Record separater now ',' for evidentia search links
• https://github.com/ngiger/oddb.org/commit/6d71ffb78af606a96e96b1c02837397d2c481074 Revert "Record separater now ',' for evidentia search links" Zeno wants that all our CSV files have ';' as delimiters
• https://github.com/ngiger/brand2csv/commit/7fea687489b8e38a5c8c2d1ca0c0dabcbdbe24c3 Updated Gemfile
• https://github.com/ngiger/brand2csv/commit/e8ad69332fb2fa454d96496337c299d84ed9e037 Write only first trademark for a given inhaber
• https://github.com/ngiger/brand2csv/commit/f0c4699866126f3d6ca9a5aeccc6b85df188c77b Updated to new rspec syntax using transpec
• https://github.com/ngiger/brand2csv/commit/dc916b179eea27ab7ddd45b936d0d97c0641cd17 Updated to use bundler/gem_task and brand2csv.gemspec
• https://github.com/ngiger/brand2csv/commit/8bc32691e19c26c2fe9396601d3e0d10984502be Prepared for release 0.3.1
• https://github.com/ngiger/brand2csv/commit/ae11f1040d1dd9b6c6ce6b55e9871052d53ea3f9 Updated to fix travis-CI builds
• https://github.com/ngiger/brand2csv/commit/284373a07a072f0d7b52ab4024038e158bfb73fa We must remove the Gemfile.lock to be able to run various variants of ruby
• https://github.com/ngiger/brand2csv/commit/9b94866c2f8c9f73071fb72c20d533b96956eef1 Relaxed some versions
• https://github.com/ngiger/brand2csv/commit/d3356a2e1c43fc609a057dd7bfc9fc7f92b85227 Force language to de_CH for travis builds
• https://github.com/ngiger/brand2csv/commit/b5cdb4c632e5b883a594d94bc40731fb99117f39 Offer a gem task like hoe

Index

• After importing Bauerfeind products we must restart the migel daemon
• brand2csv must eliminate all duplicates
• search links not correctly updated

After importing Bauerfeind products we must restart the migel daemon

Similar code is found in oddb.org/src/util/exporter.rb. But this code does not really restart the service. Must find a way to the the PID of the running migeld.

Resolved with commit Restart migel server after bauerfeind import

brand2csv must eliminate all duplicates

The spec test don't run anymore as a stub is missing for

       stub_request(:post, "https://www.swissreg.ch/srclient/faces/jsp/trademark/sr30.jsp").
         with(:body => {"autoScroll"=>"", "id_swissreg:_idcl"=>"", "id_swissreg:_link_hidden_"=>"", "id_swissreg:mainContent:id_sub_options_result:id_ckbTMChoice"=>"tm_lbl_app_date", "id_swissreg:mainContent:id_sub_options_result:sub_fieldset:id_cbxHitsPerPage"=>"250", "id_swissreg:mainContent:scroll_1"=>"", "id_swissreg:mainContent:vivian"=>"TRADEMARK REGISTER SEARCH TIMES: QUERY=[14] SELECT=[20] SERVER=[37] DELEGATE=[46] (HITS=[3])", "id_swissreg_SUBMIT"=>"1", "javax.faces.ViewState"=>"rO0ABXVyABNbTGphdmEubGFuZy5PYmplY3Q7kM5YnxBzKWwCAAB4cAAAAAN0AAE0cHQAFy9qc3AvdHJhZGVtYXJrL3NyMzAuanNw", "tmMainId"=>""},                        
              :headers => {'Accept'=>'*/*', 'Accept-Charset'=>'ISO-8859-1,utf-8;q=0.7,*;q=0.7', 'Accept-Encoding'=>'gzip,deflate,identity', 'Accept-Language'=>'en-us,en;q=0.5', 'Connection'=>'keep-alive', 'Content-Length'=>'917', 'Content-Type'=>'application/x-www-form-urlencoded', 'Host'=>'www.swissreg.ch', 'Keep-Alive'=>'300', 'Referer'=>'https://www.swissreg.ch/srclient/faces/jsp/trademark/sr3.jsp', 'User-Agent'=>'Mozilla/5.0 (X11; Linux x86_64; rv:16.0) Gecko/20100101 Firefox/16.0'}).                                                                                                                                               
         to_return(:status => 200, :body => "", :headers => {})

Running the following bundle exec bin/brand2csv --swiss_only 1.1.2016-27.6.2016, which takes quite some time. Then I created two files sorted and sorted.uniq using the following commands

• cat 1.1.2016-27.6.2016.csv | cut --delimiter=';' -f 2 | cut --delimiter='/' -f 1 | sort >sorted
• cat 1.1.2016-27.6.2016.csv | cut --delimiter=';' -f 2 | cut --delimiter='/' -f 1 | sort |uniq >sorted.uniq

Then compared the two files.

Found duplicates entry for nekahyd, number 53792/2016, which appears twice in the CSV nekahyd;53792/2016;Kalkfabrik Netstal AG, 8754 Netstal;Schweiz;Ja;25.03.2016;Kalkfabrik Netstal AG;;;;;8754;Netstal

;53792/2015;Regula Egli, Rabbentalstrasse 51, 3013 Bern;Schweiz;Nein;28.04.2016;Regula Egli;Rabbentalstrasse 51;;;;3013;Bern

Or for 00177 I get 1.1.2016-27.6.2016.csv:gran espíritu;00177/2016;Valentin Schiess, Delsbergerallee 40, 4053 Basel;Schweiz;Ja;27.04.2016;Valentin Schiess;Delsbergerallee 40;;;;4053;Basel 1.1.2016-27.6.2016.csv:CRESTA RUN;00177/2013;St. Moritz Tobogganing Club, Cresta Office, Kulm Hotel, Via Veglia 18, 7500 St. Moritz;Schweiz;Ja;03.05.2016;St. Moritz Tobogganing Club;Cresta Office;Kulm Hotel;Via Veg

But are these entries correct? It looks for me as if CRESTA RUN;00177/2013 is not correct, nor is jpeg link for Regula Egli, as she has trademarkk for "genialRegional", which does not look related to the nekahyd from the Kalkfabrik.

Did swissreg change somehow their website?

The purpose of this gem it to fetch all addresses which have at least on trademark associated with it. Therefore we want to the column C (aka inhaber) to be uniq.

It was easy and quick to create a failing spec test for this feature. Now making it pass. Various update to use brand2csv.gemspec and run in on travis-CI. See commits

• Updated Gemfile
• Write only first trademark for a given inhaber
• Updated to new rspec syntax using transpec
• Updated to use bundler/gem_task and brand2csv.gemspec
• Prepared for release 0.3.1
• Updated to fix travis-CI builds
• We must remove the Gemfile.lock to be able to run various variants of ruby
• Relaxed some versions
• Force language to de_CH for travis builds
• Offer a gem task like hoe

search links not correctly updated

The evidentia search link were not correctly updated, as Evidentia change the column seperator from ';' to ','. Fixed with commit Record separater now ',' for evidentia search links

But Zeno wants, that all our CSV files have ';' as delimiter. Therefore I will revert the commit with Revert "Record separater now ',' for evidentia search links"