Debian Question posted in
The oficial documentation can be found here.

Logstash not loading due to invalid byte sequence in UTF-8 – Debian

I have recently taken over running a Logstash system which runs on debian 9. The previous owner had installed an older version of Logstash and has left incomplete documentation on the project. I have successfully configured Logstash 7.2 locally on windows 10 and have tried to transfer this across to the Debian system replacing the necessary paths etc. I’m comming up against the following error and despite hours searching for a clue I’m left scratching my head. Any pointers would be appreciated!

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/home/user/logstash/logstash-7.2.0/logstash-core/lib/jars/jruby-complete-9.2.7.0.jar) to field java.io.FileDescriptor.fd
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Thread.exclusive is deprecated, use Thread::Mutex
Sending Logstash logs to /home/user/logstash/logstash-7.2.0/ which is now configured via log4j2.properties
[2020-07-21T08:04:35,773][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2020-07-21T08:04:35,781][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.2.0"}
[2020-07-21T08:04:37,165][INFO ][logstash.outputs.jdbc    ] JDBC - Starting up
[2020-07-21T08:04:37,195][INFO ][com.zaxxer.hikari.HikariDataSource] HikariPool-1 - Starting...
[2020-07-21T08:04:45,302][INFO ][com.zaxxer.hikari.HikariDataSource] HikariPool-1 - Start completed.
[2020-07-21T08:04:45,404][ERROR][logstash.javapipeline    ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<ArgumentError: invalid byte sequence in UTF-8>, :backtrace=>["org/jruby/RubyRegexp.java:1113:in `=~'", "org/jruby/RubyString.java:1664:in `=~'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:72:in `block in add_patterns_from_file'", "org/jruby/RubyIO.java:3329:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:70:in `add_patterns_from_file'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:403:in `block in add_patterns_from_files'", "org/jruby/RubyArray.java:1792:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:399:in `add_patterns_from_files'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:279:in `block in register'", "org/jruby/RubyArray.java:1792:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:275:in `block in register'", "org/jruby/RubyHash.java:1419:in `each'", "/home/user/logstash/logstash-7.2.0/vendor/bundle/jruby/2.5.0/gems/logstash-filter-grok-4.0.4/lib/logstash/filters/grok.rb:270:in `register'", "org/logstash/config/ir/compiler/AbstractFilterDelegatorExt.java:56:in `register'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:192:in `block in register_plugins'", "org/jruby/RubyArray.java:1792:in `each'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:191:in `register_plugins'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:463:in `maybe_setup_out_plugins'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:204:in `start_workers'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:146:in `run'", "/home/user/logstash/logstash-7.2.0/logstash-core/lib/logstash/java_pipeline.rb:105:in `block in start'"], :thread=>"#<Thread:0x1bda40f7 run>"}
[2020-07-21T08:04:45,422][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", :backtrace=>nil}
[2020-07-21T08:04:45,553][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2020-07-21T08:04:50,602][INFO ][logstash.runner          ] Logstash shut down.
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    solved by adding ":ISO-8859-1:UTF-8" to grok-pure.rb:72

    file = File.new(path, "r:ISO-8859-1:UTF-8")

    I later noticed that the file encoding of the patterns file was set to text/plain; charset=us-ascii through the command "file -bi file_name". Setting this to UTF8 may also have had an impact.


  2. 0

    the issue is because you should not have any other file under patterns_dir except grok patterns. I was having some rpm in that folder that caused the issue

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search