There's enough for everyone

गते गते पारगते पारसंगते बोधि स्वाहा गते गते पारगते पारसंगते बोधि स्वाहा

Do it yerself, Rube!

I presented this talk at the rubyfuza 2013 conference. /^\ to Marc and everyone else.

The written version of the talk as a PDF. Sorry, you had to be there to get chocolates ;–) Do it yerself, Rube!.

The Cog on youtube. Worthwhile for the soundtrack.

And the code snippets, which I didn’t have time for:

Single-level

Fetch a hash from yaml and provide dot-notation access

1
2

config_path = Pathname( '~/.config/funky_monkey.yml' ).expand_path
$settings = OpenStruct.new( YAML.load_file( config_path ) )

Encrypted / Signed config.rb

Allows you to trust your executable configuration files. Either by signing or encrypting them.

config.rb
1
2
3
4
5
6
7
8

module AppSettings
  def self.username; 'slojo'; end
  def self.password; 's3kr1t'; end
end

module GemSettings
  def self.max_worker_memory; 100.megabytes; end
end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

require 'gpgme'

class Numeric
  def megabytes; (self * 1024**2).round; end
end

# When the file is encrypted
def load_encrypted
  GPGME::Crypto.new.instance_eval do |crypto|
    config_io = Pathname 'config.rb.gpg'
    eval crypto.decrypt( config_io ).to_s, binding, config_io
  end
end

# For signing only, with plain text config file and signature
def load_signed( text_path = 'config.rb', sig_path = text_path + '.sig' )
  text_io = Pathname text_path
  signature_io = Pathname sig_path
  GPGME::Crypto.new.instance_eval do |crypto|
    crypto.verify( signature_io, signed_text: text_io ) do |signature|
      signature.valid? || raise( "invalid signature for #{text_io}")
    end
  end
  # raise will skip this
  eval text_io.read, binding, text_path
end

$SAFE = 1

How to load from untainted file names

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

def config_path
  @config_path ||= Pathname 'config.rb'
end

def restricted_config_path
  @restricted_config_path ||= Pathname 'restricted_config.rb'
end

# total trust
load config_path

def load_mostly_trusted
  lambda do
    # config file can define methods and do metaprogramming, but can't do some other stuff
    $SAFE = 1
    require config_path.to_s
  end.call
end

$SAFE = 4

Loading untrusted code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

# This fails because of "SecurityError: Insecure operation - safe_load"
# Maybe a Ruby bug?
def safe_load
  # call before $SAFE change to set instance variable
  config_path
  lambda do
    $SAFE = 4
    load restricted_config_path, true
  end.call
end

# Very restricted
def safe_eval
  # call before $SAFE change to set instance variable
  config_path
  lambda do
    unsafe_string = restricted_config_path.read
    $SAFE = 4
    eval unsafe_string, binding, restricted_config_path
  end.call
end

# could use ThreadGroup and enclose to prevent spawning of threads
def no_dos_eval
  config_path
  thread = Thread.new do
    unsafe_string = restricted_config_path.read
    $SAFE = 4
    eval unsafe_string, binding, restricted_config_path
  end
  # get thread value if it hasn't run for too long, otherwise kill it
  if thread.join(1)
    thread.value
  else
    thread.kill
  end
end

And some of the things you can and cannot do while running under $SAFE = 4

restricted_config.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

# Can't set constants
# class NewClass
#   def initialize
#     File.read '/etc/passwd'
#   end
# end

myclass = Class.new do
  def hello
    'world'
  end
end

my_instance = myclass.new
# can't even call STDOUT.write
# puts my_instance.hello

values = {
  sequel_connection: "mysql2://root:pass@localhost:/stuff",
  redis: {"host"=>"securitas"},
  redis_max_memory: 3.gigabytes,
  worker: {"max_memory"=>75.megabytes},
  output_dir: "/var/data/lenep/output",
  charset: "utf8mb4",
  collation: "utf8mb4_unicode_ci",
}

values[:friendly_name] =
case Settings.host_name
when 'production'
  'Live'
when 'uat'
  'UAT'
when 'staging'
  'Staging'
end

values

# reopen fails
# class Settings
#   def sequel_connection
#     `rm -rf /`
#   end
# end

dot-notation for a Hash

This is a nice way to see the distinction between class-oriented languages (eg c++,Java) and object oriented languages (eg Ruby) where any instance can have behaviour different to that of the other instances of the class it belongs to.

Also an example of the Class abstraction leaking.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

module HashDot
  def method_missing(meth, *args, &blk)
    property = meth.id2name
    assign = property.chomp!('=')
    super unless keys.include?( property.to_sym )
    if assign
      self[property.to_sym] = args.first
    else
      rv = self[property.to_sym]
      rv.extend HashDot if rv.kind_of? Hash
      rv
    end
  end

  def self.extended( some_hash )
    some_hash.each_pair do |k,v|
      raise "key #{k} is not a symbol" unless k.is_a? Symbol
    end
  end
end

class Numeric
  def kilobytes; (self * 1024**1).round; end
  def megabytes; (self * 1024**2).round; end
  def gigabytes; (self * 1024**3).round; end
end

some_hash = {
  sequel_connection: "mysql2://root:pass@localhost:/stuff",
  redis: { host: "fast_box", max_memory: 55.megabytes },
  worker: { max_memory: 75.megabytes },
  output_dir: "/var/data/output",
  charset: "utf8mb4",
  collation: "utf8mb4_unicode_ci",
  # 'oops' => 'value',
}.extend(HashDot)

some_hash.worker.max_memory

Read from environment variables, similar to HashDot

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

module MergeEnv
  # single-level only, ie no nested values
  def merge_environment
    keys.each do |key|
      key = key.to_s
      self[key.to_sym] = ENV[key] if ENV[key]
      self[key.to_sym] = ENV[key.upcase] if ENV[key.upcase]
    end
  end
end

some_hash = {
  sequel_connection: "mysql2://root:pass@localhost:/stuff",
  redis: { host: "fast_box", max_memory: 55000000 },
  worker: { max_memory: 75000000 },
  output_dir: "/var/data/output",
  charset: "utf8mb4",
  collation: "utf8mb4_unicode_ci",
  # 'oops' => 'value',
}.extend(MergeEnv)

# Assuming ENV['CHARSET'] == 'CESU-8'

some_hash.merge_environment
some_hash[:charset]
=> CESU-8

reload on change using rb-inotify

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

require 'rb-inotify'
require 'pathname'
require 'yaml'

# Extending a hash is not the best way to do this. Delegator?
module Reloader
  def []( *args ); maybe_reload; super; end
  def inspect; maybe_reload; super; end
  # ... etc

  def inotify_reload( config_path )
    config_path = Pathname( config_path ).realpath

    @notifier.close if @notifier
    @notifier = INotify::Notifier.new

    # have to watch the directory, mainly because vim will rename the .swp file on save...
    @notifier.watch config_path.parent.to_s, :close_write, :moved_to do |event|
      # ... so then we have to filter by event filename
      if Pathname( event.absolute_name ) == config_path
        merge! YAML.load_file( config_path )
      end
    end
  end

  protected

  def maybe_reload
    return unless @notifier
    # reload file (via notifier.watch block, above) if there is a pending change
    # Return immediately if there isn't one.
    # JRuby WARNING rb-inotify 0.8.8 docs say this does not work
    @notifier.process while IO.select( [@notifier.to_io], [], [], 0 )
  end
end

config_path = Pathname(__FILE__).parent + 'config.yml'

settings = YAML.load_file config_path
settings.extend(Reloader).inotify_reload config_path

Global functions in Ruby

Every time I do a talk, I learn something. In this case it was these global functions. String, Integer, Float, Array are implemented like this. Also Pathname. Obviously not a good thing to do unless you have objects working at that level.

1
2
3
4
5
6
7
8
9
10

require 'ostruct'

module Kernel
private
  def OpenStruct( *args )
    OpenStruct.new *args
  end
end

OpenStruct( key: 'value' )

The private is there because otherwise you could do something like "banjo".OpenStruct( key: 'value' ) which would work, but which doesn’t make much sense.

Comments