20160718-fix-search-filter-sandoz-bbmb-ch-fix-import-error-encoding-issue-and-flavor-for-sandoz-xmlconv-bbmb-ch

<< 20160719-fix-import-error-sandoz-xmlconv-ch-and-remove-flavor-bbmb-ch | Index | 20160715-debug-encoding-issuen-sandoz-bbmb-ch >>

Summary

• Setuped sandoz.bbmb.ch, sandoz.xmlconv.bbmb.ch in local.
• Fixed search filter on sandoz.bbmb.ch
• Updated sbsm version on production (sandoz.bbmb.ch)
• Removed `Thread.exclusize` (deprecated) and then replaced Mutex
• Implemented (missing) sorting for Aktiviert and Last Login columns (sandoz.bbmb.ch)
• Debugging importer error on sandoz.xmlconv.bbmb.ch

Commits / Patches / Pull Requests, Scripts

• Fix search filter [commit, bbmb]
• Fix search filter [pull request, bbmb]
• Use Mutex and remove Thread.exclusive [commit, bbmb]
• Fix executable command permission (to 755) [commit, bbmb]
• Add missing sorting for valid and last_login [commit, bbmb]
• Replace Thread.exclusive with Mutex#synchronize[commit, bbmb]
• Created new version [pull request, bbmb]

Index

• Fix encoding issue for search filter on sandoz.bbmb.ch
• Remove thread exclusive
• Fix executable command permission
• Fix sort order
• Fix RuntimeError by importer on sandoz.bbmb.ch

Fix encoding issue for search filter on sandoz.bbmb.ch

Setup applications

Before fixing issues, I've setup following bbmb applications in my local.

• sandoz.xmlconv.bbmb.ch
• sandoz.bbmb.ch
• vetoquinol.bbmb.ch
• virbac.bbmb.ch

Search filter

On production, In search filter, we've got followig error.

@4000000057888f590e569dbc error in SBSM::Session#http_headers: /de
@4000000057888f590e56a1a4 ArgumentError
@4000000057888f590e56a1a4 invalid byte sequence in UTF-8
@4000000057888f590e56a1a4
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/html/state/customers.rb:29:in
`match'
@4000000057888f590e56a58c
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/html/state/customers.rb:29:in
`block (3 levels) in init'
@4000000057888f590e56a58c
/usr/local/lib/ruby/gems/2.3.0/gems/bbmb-2.0.5/lib/bbmb/html/state/customers.rb:29:in
`any?'
...

I think this Invalid byte sequence comes from sbsm-1.2.3's input (installed on Ruby 1.8.6 for mod_ruby). And some values are empty. Then I've added next 2 changes.

Remove thread exclusive

• Change sbsm to sbsm-1.2.8 (newest) in apache config file (mod_ruby is compiled against 1.8.6, but use sbsm installed for application in Ruby 2.3.1)
• Skip empty string
• Avoid error by enexpected input

Commit: Fix search filter

Remove thread exclusive

I've got deprecated warning in test or log.
So I've updated Thread usage with Mutex.

% bundle exec test/suite.rb                                                     
bash -c 'bundle exec /path/to/bbmb/test/model/test_customer.rb --seed 62766  2>&1 | \
  tee test_customer.log; ( exit ${PIPESTATUS[0]} )'
Run options: --seed 62766                                                          

# Running:                                                                         

....Thread.exclusive is deprecated, use Mutex                                      
/path/to/bbmb/lib/bbmb/model/customer.rb:75:in `inject_order'                   
/path/to/bbmb/test/model/test_customer.rb:66:in `test_inject_order'             
/path/to/bbmb/.bundle/gems/ruby/2.3.0/gems/minitest-5.9.0/lib/minitest/test.rb:107:in `block (3 levels) in run'
/path/to/bbmb/.bundle/gems/ruby/2.3.0/gems/minitest-5.9.0/lib/minitest/test.rb:204:in `capture_exceptions'
/path/to/bbmb/.bundle/gems/ruby/2.3.0/gems/minitest-5.9.0/lib/minitest/test.rb:104:in `block (2 levels) in run'
...

Commits:
* Use Mutex and remove Thread.exclusive

• Replace Thread.exclusive with Mutex#synchronize

Fix sort order

For some fields, Sort feature does not work.
It seems that thees objects are Boolean and Date.

could not sort by valid
could not sort by valid
could not sort by last_login
...

It seems that this sort fails, because
these columns are result of following methods (with argument user.email).
Yus has these methods.

• user.entity_valid?(user.email)
• last_login(user.email)

I made sort feature for this columns. It works.
But, Its are very slow, because by each line, sorting needs query to Yus Server (session). In my local machine it takes 12 - 15 seconds...
Perhaps, By above reason, the sorting for these 2 columns was not implemented.
Hmm, How should I avoid this?

I've used Delegator and cached some value into instance variable, then reduced queries to Yus server.
But, (In the future) It might need cache store (etc).

Commit: Add missing sorting for valid and last_login

Fix executable command permission

In the installation by gem command, these commands are appended x permission automatically.
But I've added this in source code for convenience.

Commit: Fix executable command permission (to 755)

Fix RuntimeError by importer on sandoz.bbmb.ch

If user access to following page, then RuntimeError occurs.
http://sandoz.xmlconv.bbmb.ch/de/transaction/state_id/70186891604580/transaction_id/29066

This is already rescued (and reported), but we have to know why this error occurs.

RuntimeError Bestellung OK, Eintrag in BBMB fehlgeschlagen: DRb::DRbConnError druby://localhost:12004 - #

/usr/local/lib/ruby/2.3.0/drb/drb.rb:745:in `rescue in block in open'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:739:in `block in open'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:738:in `each'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:738:in `open'
/usr/local/lib/ruby/2.3.0/drb/drb.rb:1248:in `initialize'
...