Skip to content
accelerando

Category Archives: English posts

Linux: Persistent wake-on-lan

03-Mar-10

Just a quick reminder for myself: You need to enable wake-on-lan on the nic in most linux distributions via ethtool.

If your nic supports WOL, it probably needs to be enabled in your computers BIOS first.

Most WOL tools use the “MagicPacket(tm)” method, so the right command to enable it on the nic “eth0″ would be

ethtool -s eth0 wol g

This is not a persistent setting and it is gone after a reboot. Most tips around recommend creating a runlevel script for executing this command after boot or before shutdown.

I’ve added the command as a hook in my network scripts:

auto eth0
iface eth0 inet dhcp
        pre-down ethtool -s eth0 wol g

You can also use post-up. I like this method because i tend to forget about my runlevel scripts and so i keep the network stuff in one place ;)

If you are a debian user, you maybe need to stop your system from turning off the nic on HALT. To do so, add

NETDOWN=no

to “/etc/default/halt”.

Oracle: Drop table if exists replacement

16-Feb-10

Mysql has a nice “if exists” addition to the drop table statement. If the table to be dropped does not exists, it doesn’t raise an exception but only creates a warning.

In Oracle RDMBS you can emulate this behavior like so:

BEGIN execute immediate 'drop table INSERT_TABLE_NAME_HERE'; EXCEPTION WHEN others THEN IF SQLCODE != -942 THEN RAISE; END IF; END;
/

Ugly, but it works very well.

An iterable array

12-Jan-10

Java has the nice Iterable interface (since Java 5, i guess) that allows object oriented loops like

List<String> strings = new ArrayList<String>();
for(String string : strings) 
	System.out.println(string);

but guess what, a simple array is not iterable…

In case you need one, feel free to use this one:

package ac.simons;
 
import java.util.Iterator;
 
public class IterableArray<T> implements Iterable<T> {
	private class ArrayIteratorImpl implements Iterator<T> {
		private int position = 0;
		@Override
		public boolean hasNext() {
			return data != null && this.position < data.length;
		}
 
		@Override
		public T next() {
			return data[position++];
		}
 
		@Override
		public void remove() {
			throw new UnsupportedOperationException();
		}		
	}
 
	private final T[] data;
 
	public IterableArray(T[] data) {
		this.data = data;
	}
 
	@Override
	public Iterator<T> iterator() {
		return new ArrayIteratorImpl();
	}	
}

Create ZIP Archives containing Unicode filenames with Java

05-Jan-10

It is harder than i thought to create a simple Zip Archive from within Java that contains entries with unicode names in it.

I’m actually to lazy to read all the specs, but it says something that the entries in a zip archive are encoded using “Cp437″. The buildin Java compressing api has nothing to offer for setting the encoding so i tried Apache Commons Compress.

The manual says the following about interop :

For maximum interop it is probably best to set the encoding to UTF-8, enable the language encoding flag and create Unicode extra fields when writing ZIPs. Such archives should be extracted correctly by java.util.zip, 7Zip, WinZIP, PKWARE tools and most likely InfoZIP tools. They will be unusable with Windows’ “compressed folders” feature and bigger than archives without the Unicode extra fields, though.

That didn’t work for me.

After some cursing, this is my solution:

final ZipArchiveOutputStream zout = new ZipArchiveOutputStream(new BufferedOutputStream(new FileOutputStream(fc.getSelectedFile())));
zout.setEncoding("Cp437");
zout.setFallbackToUTF8(true);
zout.setUseLanguageEncodingFlag(true);								
zout.setCreateUnicodeExtraFields(ZipArchiveOutputStream.UnicodeExtraFieldPolicy.NOT_ENCODEABLE);

I specifying explicitly the encoding but instead of using utf-8, which didn’t work for my utf-8 strings (wtf??), i’m using the Cp437 from the specs and some other magic options and it works for me in 7zip, WinZip and even Windows’ “compressed folders”.

Edit: Unfortunately, in Mac OS X’s Unzip utility, the non Cp437 are broken. If anyone has a good idea, feel free to leave a comment.

Today: Fun with Unicode, Regex and Java.

03-Nov-09

Some would say, i have 3 problems ;)

private final static Pattern placeholder = Pattern.compile("#\\{(\\w+?)\\}");

won’t match “Mot#{ö}rhead” for example.

To replace the word character \w you either need the list of possible unicodeblocks like [\p{InLatin}|\p{InEtc}] (you get the codes for the blocks through “Character.UnicodeBlock.forName” or you’re lazy like me and just use the dot:

private final static Pattern placeholder = Pattern.compile("#\\{(.+?)\\}");

Oh what a day… :/

Redcloth / Textilize :hard_breaks is broken in Rails 2.3.4

24-Sep-09

The default behavior for quite a long time was :hard_breaks in the textilize helper method in rails.

Hard Breaks means: All line breaks are turned into <br />’s.

Somebody changed the textilize helper in “actionpack-2.3.4/lib/action_view/helpers/text_helper.rb” in 2.3.4 and added the ability to pass some options but broke the default behavior here:

def textilize(text, *options)
        options ||= [:hard_breaks]
        # ...

Options will never be null.

I fixed this by monkey patching the module through the following code in config/initializers/textilizepatch.rb

module ActionView
  module Helpers
    module TextHelper
      def textilize(text, *options)
        options = [:hard_breaks] if options == nil || options.size == 0
 
        if text.blank?
          ""
        else
          textilized = RedCloth.new(text, options)
          textilized.to_html
        end
      end
    end
  end
end

Such changes should be tested… *grml*

SVN: Revert to previous version

09-Sep-09

Just a quick reminder:

To revert a complete working copy or a single file use:

svn merge -rHEAD:PREV .
# or
svn merge -rHEAD:PREV path/to/file
svn commit -m "reverted"

Logging in Rails outside a controller

25-Aug-09

You can use

RAILS_DEFAULT_LOGGER.log "foobar"
# or
Rails.logger.log "blah"

outside a controller for logging.

Creating a self containing mvc application with Sinatra

29-Jul-09

I recently presented biking.michael-simons.eu, a Sinatra based application written in Ruby.

Its main purpose for me is to keep track of my milage in 2009.

Although the application is completely self containing it has some nice features:

Simple setup

The application is simple and i didn’t want to use a “big” database like PostgreSQL or MySQL, therefore i choose SQLite. Together with DataMapper i can use the following and have a running SQLite database in no time:

configure :development do
  DataMapper.setup(:default, "sqlite3://#{Dir.pwd}/biking.dev.sqlite3")
end
 
# A milage is stored for a bike at the beginning of the month.
# Its the milage of the bike at this point of time.
class Milage
  include DataMapper::Resource
  property :id,          Integer, :serial => true
  property :when,        Date, :nullable => false
  property :value,       BigDecimal, :nullable => false, :precision => 8, :scale => 2
  property :created_at,  DateTime
 
  belongs_to :bike
 
  validates_is_unique :when, :scope => :bike
 
  is :list, :scope => [:bike_id]
end
 
DataMapper.auto_upgrade!

Creating charts with the google_chart gem

Google has a nice api for creating charts, the Google Charts API. The charts are created through URL parameters and as we know, most browsers limits the length of a query string. And for that, the parameters must be encoded.

Nobody wants to do it themselves so there is gchartrb that i use:

  # Create the google chart
  # All will include all float values
  all = Array.new
  @gc = GoogleChart::BarChart.new('800x375', nil, :vertical, false) do |bc|
    @bikes.each do |b|
      # Help is an array of floats for all periods
      hlp = b.milage_report.collect{|mr| mr[:milage]}
      # and gets added to the chart for that bike
      bc.data b.name, hlp, b.color
      all += hlp
    end
 
    hlp = Array.new
    # Get all periods. It sure can be done via a group by but 
    # the app should be database agnostiv
    periods = Milage.all.collect{|m| m.when.strftime '%m.%Y'}.uniq
    # Get all other trips in this period and also add them to the chart
    periods.each {|period| hlp <<= (AssortedTrip.sum(:distance, :conditions => ['strftime("%m.%Y", "when") = ?', period]) || 0.0)}
    all += hlp
    bc.data "Assorted Trips", hlp[0,hlp.length-1], "003366"
 
    # Define the labels
    bc.axis :x, :labels => periods, :font_size => 16, :alignment => :center
    # and the range from 0 to the highest value
    bc.axis :y, :range => [0,all.max], :font_size => 16, :alignment => :center
    bc.show_legend = true
    bc.grid :x_step => 0, :y_step => (100.0/all.max)*25.0, :length_segment => 1, :length_blank => 0 if all.size > 0
  end if @bikes.size > 0

The gc object will be used as simple as

%img{:src => @gc.to_url, :alt => 'gc'}

Using libxml for parsing RSS feeds

It seems that i’m not going to write an application without including my daily faces project dailyfratze.de in some way.

So i decided not only to present numbers but also my biking pictures. The images are available through a Media RSS feed. The feed itself contains pointers to other pages of that very same feed.

One of the fastest ways to parse XML is LibXML and luckily, it’s available for ruby through LibXML Ruby.

Together with memcached it can be used to efficiently handle feeds like so:

def BikingPicture.random_url
  # Check if pictures are cached...
  biking_pictures = @@cache['biking_pictures']
  unless biking_pictures then
    # Start retrieving the feed
    url = @@uri.parse("http://dailyfratze.de/michael/tags/Thema/Radtour?format=rss&dir=d")
    next_page = false
 
    Net::HTTP.new(url.host).start do |http|
      # Its a media rss feed that defines previous and next feeds
      while url
        req = Net::HTTP::Get.new("#{url.path}?#{url.query}")
        xml = http.request(req).body
        # Parse the data
        doc = LibXML::XML::Parser.io(StringIO.new(xml)).parse
        # Unless this url hasn't been retrieved...          
        unless @@cache[url.to_s]            
          doc.find('/rss/channel/item').each do |item|
            # Get all pictures and store them if not already grapped
            biking_picture = BikingPicture.first :url => item['url']
            unless biking_picture
              biking_picture = BikingPicture.new({:url => item.find_first('media:thumbnail')['url'], :link => item.find_first('link').content })
              biking_picture.save
            end
          end
          # Mark this url as seen
          @@cache[url.to_s] = next_page
          next_page = true                          
        end
        # Check if there are more feeds...
        next_url = doc.find_first("/rss/channel/atom:link[@rel = 'next']")
        url = if next_url
          @@uri.parse next_url['href']
        else
          nil
        end
      end
    end
    # Get the data...
    biking_pictures = BikingPicture.all()
    # ...and store it
    @@cache.set 'biking_pictures', biking_pictures, 3600
  end
  biking_pictures.sort_by{rand}[0]
end

Sinatra, Passenger and Memcached

Speaking of being efficient, the application certainly runs through Phusion Passenger aka modrails.

Running a Sinatra app under passenger is as simple as this: Create a directory for a new vhost, setup a structure like

biking
\- public
 - tmp
 - config.ru
 - biking.rb

and let config.ru contain the following:

root_dir = File.dirname(__FILE__)
 
require 'biking.rb'
 
set :environment, ENV['RACK_ENV'].to_sym
set :root,        root_dir
set :app_file,    File.join(root_dir, 'biking.rb')
disable :run
 
run Sinatra::Application

The apache vhost is configured like so:

<VirtualHost *>
        DocumentRoot "/path/to/the/applications/public/folder"
        RackBaseURI /
</VirtualHost>

I have turned off the Rack and Rails Autodetect features (RailsAutoDetect off, RackAutoDetect off) so i need to explicitly turn them on for a vhost.

After that, the application is running and thats all there is.

But wait. I’ve written before about problems with memcache-client and Passenger and this problems needs to be addressed in a Rails as well as Rack application.

I handle them in Sinatra as follows:

configure do
  # Create a global memcache client instance...
  @@cache = MemCache.new({
    :c_threshold => 10000,
    :compression => true,
    :debug => false,
    :namespace => 'some_ns',
    :readonly => false,
    :urlencode => false
  })
  @@cache.servers = 'some_server:11211'
 
  if defined?(PhusionPassenger)
    PhusionPassenger.on_event(:starting_worker_process) do |forked|
      @@cache.reset if forked
    end
  end
end

If PhusionPassenger is available, install an event handler that resets the freshly forked memcached connection to avoid corruption.

Using Geonames.org

Google Maps is great but one thing that’s often forgotten is reverse geocoding. Google doesn’t offer such api (as far as i know) but GeoNames does.

I’ve written a mobile J2ME client around JSR 179 api that can push my current location to this server when i’m biking so my girlfriend can follow my rides on the map.

The client pushes latitude and longitude and the server replies with the coordinates in angles and the name of the place. The name is retrieved through the Ruby Geonames API like so:

#
# Adds a new location
#
post '/locations' do
  require_administrative_privileges
  location = Location.new params
  if location.save then
    begin
      country_subdivision  = Geonames::WebService.country_subdivision(location.latitude, location.longitude)
      places_nearby = Geonames::WebService.find_nearby_place_name(location.latitude, location.longitude).first
      (location.description = "#{places_nearby.name}, #{country_subdivision ? country_subdivision.admin_name_1 + ', ' : ''}#{places_nearby.country_name}"[0,2048].strip) and location.save if places_nearby
    rescue Exception => exc      
    end
  end
  "#{location.to_s}\n"
end

Inline templates, external resources

The thing started as a small application. PHP was out of the question but Rails was overkill, too. I just wanted one single file.

I can use the great Haml syntax as inline templates. This goes for creating HTML as well as for CSS through Sass.

To get some basics, i also include one of the W3 core styles.

The only resources that lives outside biking.rb is JQuery for easy creation of the autorefreshing code and some images, namely the cute green bikers from Greensmilies.

Summary

It’s absolutely possible to write an application with thee MVC Pattern in mind without one of the “big” frameworks in just one single script.

I would go much further with this app using just one script, but for it’s current purpose, it’s great, i think.

I hope you enjoyed reading my little annotations as i enjoyed writing them and the application. The full source code is available right through biking.michael-simons.eu and it’s published under the BSD license.

Javas String.replaceAll

21-Jul-09

The following statement

"foo baz".replaceAll("baz","$bar");

will present you an java.lang.IllegalArgumentException: Illegal group reference exception as the replacement string can contain backreferences to the search pattern as stated in Mather#replaceAll:

This method first resets this matcher. It then scans the input sequence looking for matches of the pattern. Characters that are not part of any match are appended directly to the result string; each match is replaced in the result by the replacement string. The replacement string may contain references to captured subsequences as in the appendReplacement method.

Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string. Dollar signs may be treated as references to captured subsequences as described above, and backslashes are used to escape literal characters in the replacement string.

Luckily, there is an easy solution:

"foo baz".replaceAll("baz",Matcher.quoteReplacement("$bar"));
Close
E-mail It