Debugging grok expressions on the terminal command line in ruby

Debugging grok expressions on the terminal command line in ruby_PHP tutorial

WBOY

Release： 2016-07-13 10:16:11

Original

923 people have browsed it

Debugging grok expressions on the terminal command line in ruby

Anyone who uses logstash knows to debug grok regular expressions on http://grokdebug.herokuapp.com. Now the question arises: Which company has the best circumvention technology? The page uses js files from the Google domain name, so access often fails. Therefore, quick debugging through the command line on the terminal has become a necessity.

In fact, when logstash was still in 1.1, there was a group of people on the official wiki who taught everyone how to interactively test grok expressions through irb. But I don’t know why the wiki page disappeared later... Fortunately, the code itself is not complicated. Just write a few lines of script and you can achieve the goal:

代码如下

#!/usr/bin/env ruby
require 'rubygems'
gem 'jls-grok', '=0.11.0'
require 'grok-pure'
require 'optparse'
require 'ap'
options = {}
ARGV.push('-h') if ARGV.size === 0
OptionParser.new do |opts|
opts.banner = 'Run grokdebug at your terminal.'
options[:dirs] = %w(patterns)
options[:named] = false
opts.on('-d DIR1,DIR2', '--dirs DIR1,DIR2', Array, 'Set grok patterns directories. Default: "./patterns"') do |value|
    options[:dirs] = value
end
opts.on('-m MESSAGE', '--msg MESSAGE', 'Your raw message to be matched') do |value|
    options[:message] = value
end
opts.on('-p PATTERN', '--pattern PATTERN', 'Your grok pattern to be compiled') do |value|
    options[:pattern] = value
end
opts.on('-n', '--named', 'Named captures only') do
    options[:named] = true
end
end.parse!
grok = Grok.new
options[:dirs].each do |dir|
if File.directory?(dir)
    dir = File.join(dir, "*")
end
Dir.glob(dir).each do |file|
    grok.add_patterns_from_file(file)
end
end
grok.compile(options[:pattern], options[:named])
ap grok.match(options[:message]).captures()

Test it:

代码如下

$ sudo gem install jls-grok awesome_print
$ ruby grokdebug.rb
Run grokdebug at your terminal.
    -d, --dirs DIR1,DIR2             Set grok patterns directories. Default: "./patterns"
    -m, --msg MESSAGE                Your raw message to be matched
    -p, --pattern PATTERN            Your grok pattern to be compiled
    -n, --named                      Named captures only
$ ruby grokdebug.rb -m 'abc123' -p '%{NUMBER:test}'
{
         "test" => [
        [0] "123"
    ],
    "BASE10NUM" => [
        [0] "123"
    ]
}
$ ruby grokdebug.rb -m 'abc123' -p '%{NUMBER:test:float}' -n
{
    "test" => [
        [0] 123.0
    ]
}

Yes, I have more type conversion functions than the grokdebug website. The jls-grok it uses is version 0.10.10, and I am using the latest version 0.11.0.