view · edit · sidebar · attach · print · history

20160714-fix-encoding-issue-sandoz-bbmb-ch

<< 20160715-debug-encoding-issuen-sandoz-bbmb-ch | Index | 20160713-migrate-davaz-com-dojo-widgets-fix-login-xmlconv-sandoz-bbmb-ch-fix-encoding-issue-sandoz-bbmb-ch >>

Summary

  • Fix encoding issue (It has still wrong code points)

Commits / Patches / Pull Requests, Scripts

Index

Debug sandoz.bbmb.ch Encoding Issue

Yesterday, I've tried as experiment via bin/admin, encoding issue to fix. See below
http://dev.ywesee.com/Yasu/20160713-migrate-davaz-com-dojo-widgets-fix-login-xmlconv-sandoz-bbmb-ch-fix-encoding-issue-sandoz-bbmb-ch#debug-sandoz-bbmb-ch-encoding-issue

Setup sandoz.bbmb.ch in local

  • Get yml config files from production (scp)
  • Create "yus_sandoz" database in PostgreSQL
  • Create "yus" user
  • Create "bbmb_sandoz" database in PostgreSQL
  • Create "sandoz" user
  • Create apache config file
  • Get backup data for (yus, application database) and Import with zcat
  • Install gems with Bundle
  • Create init (boot) script (or Use daemontools)

Create yus database for sandoz

Create yus database and its role as you like. 

postgres=# CREATE DATABASE "yus_sandoz" WITH ENCODING 'UTF8' TEMPLATE = template0;
CREATE DATABASE
postgres=# ALTER DATABASE "yus_sandoz" OWNER TO "yus";
ALTER DATABASE

Download data from production

In production has another name as database and role.
Replace it with regexp in your editor as you like before import
(e.g. `:%s/Owner:\ssandoz/Owner: yus/g` and `:%s/OWNER\sTO\ssandoz/OWNER TO yus/g`) 

% scp example.org:/path/to/backups/postgresql/July-2016/2016-07-13/22\:00-postgresql_database-sandoz_yus-backup.gz .
Password: 
22:00-postgresql_database-sandoz_yus-backup.gz                                               100%   47KB  46.9KB/s   00:00

Import data

% zcat 22:00-postgresql_database-sandoz_yus-backup.gz | psql -U yus yus_sandoz

Create application database

sandoz.bbmb.ch uses odba (postgresql) as database.
So setup database like yus above. 

postgres=# CREATE ROLE "sandoz";
CREATE ROLE
postgres=# ALTER ROLE sandoz WITH PASSWORD 'xxxxx';
ALTER ROLE
postgres=# ALTER ROLE sandoz CREATEDB REPLICATION LOGIN;
ALTER ROLE

% zcat 22:00-postgresql_database-bbmb_sandoz-backup.gz | psql -U sandoz bbmb_sandoz

bbmbd can't connect into PostgreSQL

I've confirmed DB user and password are correct, but bbmbd fails to connect to db. 

% ./run-bbmbd.sh
...
bundler: failed to load command: bbmbd (/path/to/sandoz.bbmb.ch/.bundle/gems/ruby/2.3.0/bin/bbmbd)
DBI::OperationalError: FATAL:  password authentication failed for user "sandoz"
...

This problem was casude in lib/persistence/odba.rb ob bbmb.
The bbmb expects password key as db_auth not db_path.

In config file (etc/config.yml) 

db_name: sandoz
db_user: foo
db_pass: foo

But bbmb expects db_auth in here:
https://github.com/zdavatz/bbmb/blob/c246cebb60055287826b67a82fab1bb87bedcd65/lib/bbmb/persistence/odba.rb#L40 

 ODBA.storage.dbi = ODBA::ConnectionPool.new("DBI:Pg:#{@config.db_name}",
                                             @config.db_user, @config.db_auth)

Fix encoding Script

I've written a small script to fix string encoding.

Commit:

Firt commit, It works on my local. But In Production needs pull request version above.
I don't know why (Though I used same data...)

Attach: screenshot-sandoz-bbmb-ch-fixed-character-encoding-20160714.png

Strategy

1. encode(ISO-8859-1) then force_encoding(UTF-8) 2. focne_encoding(ISO-8859-1) then encode(UTF-8) (if 1st fails) 

# i.e. force_encoding(Encoding::ISO_8859_1) and encode(Encoding::UTF_8)
ch.bbmb.sandoz> c = BBMB::Model::Customer.find_by_customer_id('4100611843'); \
puts c.instance_variables.map {|var| [var, c.instance_variable_get(var)] }.select { \
|k, v| v.is_a?(String) ? [k, v] : nil }.compact.map { \
|k, t| [k, t.force_encoding(Encoding::ISO_8859_1).encode(Encoding::UTF_8)] }.map { \
|var, val| c.instance_variable_set(var, val) }; c.odba_store
-> #<BBMB::Model::Customer:0x007fb8839aa630>

Non flavored url issue again

It seems that My pull request in yesterday night is wrong. In my local, sandoz.bbmb.ch works without this pull request. And it uses flavor bbmb

So URL must have path /de/bbmb/customer/customer_id/4100609297 instead of /de/customer/customer_id/4100609297.

But sandoz.bbmb.ch on production, application does not work this pull request.
Setup of sandoz.bbmb.ch on production might have something another wrong...

Commit is:
https://github.com/zdavatz/sandoz.bbmb.ch/commit/8f8f54deb4697150366c81d4199baccb201ab265

And even if I put DEFAULT_FLAVOR sandoz in apache config file, then sandoz.bbmb.ch use bbmb as flavor.
This is also strange. Where does it come from?

I've found here:
https://github.com/zdavatz/bbmb/blob/45f4bbaef3d4c42889eb4422e3dfaa06fee98d77/lib/bbmb/html/util/session.rb#L20

Yesterday, On Production, Its flavor was sbsm, and it does not work even if I gave sandoz or bbmb.

BBMB boot script (daemontools run) does not use bundler

I've found a little strange situation on production.
The run scipt does not use bundler, and I cannot find Gemfile.lock.

This might be Problem.

Hmmm, But Production works now. (without Bundler)
So, I will consider this issue on next week.

In my local, sandoz.bbmb.ch needs this patch (without my pull request...) 

% git diff lib/
diff --git a/lib/bbmb/html/util/lookandfeel.rb b/lib/bbmb/html/util/lookandfeel.rb
index acee895..64b19ef 100644
--- a/lib/bbmb/html/util/lookandfeel.rb
+++ b/lib/bbmb/html/util/lookandfeel.rb
@@ -200,13 +200,13 @@ La versione aggiornata delle nostre CGC puņ essere visionata e <a href='http://
   alias :orig_base_url :base_url

   # Provides non flavored base url
-  def base_url
-    _flavor = @flavor
-    @flavor = nil
-    url = orig_base_url
-    @flavor = _flavor
-    url
-  end
+  #def base_url
+  #  _flavor = @flavor
+  #  @flavor = nil
+  #  url = orig_base_url
+  #  @flavor = _flavor
+  #  url
+  #end
 end
     end
   end
view · edit · sidebar · attach · print · history
Page last modified on July 14, 2016, at 08:34 PM