Last Modified
2014-06-06 13:41:46 +0000
Requires

Description

BufferedTokenizer takes a delimiter upon instantiation, or acts line-based by default. It allows input to be spoon-fed from some outside source which receives arbitrary length datagrams which may-or-may-not contain the token by which entities are delimited.

By default, new BufferedTokenizers will operate on lines delimited by ā€œnā€ by default or allow you to specify any delimiter token you so choose, which will then be used by String#split to tokenize the input data

@example Using BufferedTokernizer to parse lines out of incoming data

module LineBufferedConnection
  def receive_data(data)
    (@buffer ||= BufferedTokenizer.new).extract(data).each do |line|
      receive_line(line)
    end
  end
end

@author Tony Arcieri @author Martin Emde