view · edit · sidebar · attach · print · history

Day Three

<< DayTwo | Index | >>

  • htmlrid crash: Approach 2 -- stare at the grid.c code and try to spot a bug. The function around grid.c:403 seems to be correct (at least when considering ~2 function call levels below), as long as the inputs are correct.
  • htmlgrid crash: Approach 1 -- try to find a request that crashes oddb.

Looked at apache error log for clusters of errors. Usually the errors were grouped in ~30 second windows, so that may mean server segfaulted and got restarted. After trying ~10 such request no luck crashing oddbd. Possibly this means the bug is non-deterministic. The other possibility is that the actual request was not logged in apache error_log/access_log.

In any case, a good idea is to log all request coming into oddbd, along with timestamps.

  • protip -- use firefox Export Cookies plugin to export cookies for use with 'wget --load-cookies'
  • crash of oddb version 2a90ba22634b1724c9e74853e2097679ed82ab0f (c7a2924264f5362e6098d449fa385c65c485e6a3 was ok) :
        /usr/local/lib/ruby/gems/1.9.1/gems/odba-1.0.8/lib/odba/marshal.rb:15:in `load': undefined class/module ODDB::Migel::Product (ArgumentError)

=> needed new database.

  • protip - piping stderr into a file (examples)
 # pipe all output into a file
 user@localhost ~/currency $ /usr/local/bin/ruby bin/currencyd &> | tee stdout_with_stderr.log
 # tee all output into a file (as usual, tee output to console may lag, due to (not) flushing inside the program)
 user@localhost ~/currency $ /usr/local/bin/ruby bin/currencyd 2>&1 | tee stdout_with_stderr.log
 # pipe stdout (unix file descriptor 1) and stderr (unix file descriptor 2) into separate files
 user@localhost ~/currency $ /usr/local/bin/ruby bin/currencyd 1>stdout.log 2>stderr.log
  • Benchmarking mod_ruby compiled against ruby 1.8 and 1.9 has similar results, ruby 1.8 seems just a little bit faster. In massively concurrent situations this builds up to more failed requests with ruby 1.9
  • note - both mod_ruby-1.3 and mod_ruby-git, when configured with /usr/local/ruby1.9, link statically to libruby

=> reinstall ruby1.9 (configure --enable-shared), and recompile mod_ruby

 $ ldd =>  (0xffffe000) => /usr/local/lib/ (0xb74f0000) => /lib/ (0xb74c6000) => /lib/ (0xb74bd000) => /lib/ (0xb74b9000) => /lib/ (0xb7487000) => /lib/ (0xb7461000) => /lib/ (0xb7303000)
 /lib/ (0xb777d000)
  • going anywhere from the oddb index pages results in error_log :
 [Fri Jan 13 10:39:22 2012] [error] mod_ruby: /usr/local/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in  require': /usr/local/lib/ruby/gems/1.9.1/gems/sbsm-1.0.3/lib/sbsm/flavored_uri_parser.rb:9: invalid multibyte char (US-ASCII) (SyntaxError)

=> add to the file

 # -*- coding: ascii-8bit -*-
  • any request now shows (the source of) /var/www/

=> 'qfile mod_ruby' shows that we are missing a configuration file. Copy the standard gentoo's 21_mod_ruby.conf ( to /etc/apache2/modules.d, then LoadModule in /etc/apache2/vhosts.d/01_oddb_vhosts.conf is no longer needed

  • security error in sbsm (apache error_log)
 [Fri Jan 13 09:23:27 2012] [error] mod_ruby: /usr/local/lib/ruby/gems/1.9.1/gems/sbsm-1.0.3/lib/sbsm/trans_handler.rb:21:in `expand_path': Insecure operation - expand_path (SecurityError)

=> replace File.dirname(__FILE__) with File.dirname(__FILE__).untaint on lines 21 and 23, add '.untaint' to 'request.server.document_root' on line 29

  • checking apache2 with mod_ruby/1.9
 $ sudo /etc/init.d/apache2 start
 * apache2 has detected an error in your setup:
 Syntax error on line 17 of /etc/apache2/vhosts.d/01_oddb_vhost.conf:
 Invalid command 'RubyAddPath', perhaps misspelled or defined by a module not included in the server configuration
 * ERROR: apache2 failed to start

=> add on top of /etc/apache2/vhosts.d/01_oddb_vhost.conf :

 LoadModule ruby_module /usr/lib/apache2/modules/

=> apache now starts, but after loading oddb main page the following error appears in /var/log/apache2/error_log :

 [Fri Jan 13 08:55:03 2012] [error] mod_ruby: failed to require sbsm/trans_handler
 [Fri Jan 13 08:55:03 2012] [error] mod_ruby: error in ruby
 [Fri Jan 13 08:55:03 2012] [error] mod_ruby: /usr/local/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': /home/user/.local/rubygems/lib/rockit/token.rb:139: invalid multibyte char (UTF-8) (SyntaxError)
 [Fri Jan 13 08:55:03 2012] [error] mod_ruby: /home/user/.local/rubygems/lib/rockit/token.rb:139: syntax error, unexpected '~', expecting ')'
 [Fri Jan 13 08:55:03 2012] [error] mod_ruby:     super("EOF", "\xa4~~\xa4\xa4~^^~" + rand(1e10).inspect)

=> as per, add magic comment:

 # -*- coding: ascii-8bit -*-

to files:

  • compiling mod_ruby against ruby1.9
 $ emerge -C mod_ruby
 $ git clone
 $ cd mod_ruby
 $ ./configure.rb # same as /usr/local/bin/ruby configure.rb
 $ make

=> gives the error:

 /usr/include/apache2/ap_config.h:25:17: fatal error: apr.h: No such file or directory

=> add '-I/usr/include/apr-1' to APACHE_INCLUDES line in Makefile, and run 'make' again, then 'sudo make install'

view · edit · sidebar · attach · print · history
Page last modified on January 19, 2012, at 01:59 PM