I've been stuck on this problem for quite some time now, any help would be gladly appreciated. So I have a docker compose containing Elastic Stack services, plus a Cassandra database. I am trying to index data from the Cassandra…
My goal is to read data from kinesis and upload csv file on s3, but in csv file arrow(=>) is coming in json data insted of colon(:) Logstash config file input { kinesis { application_name => "logstash" kinesis_stream_name => "events"…
I have installed ELK on one of the VM(Ubuntu machine) in Azure Enviroment, i am unable to access it via http://localhost:5601 in a sandbox machine which is in the same environment how to login to the ELK stuff do I…
I have a backend production with microservice architecture that I wrote with nestjs. All services here are running as separate docker containers. In addition to these services, in my docker-compose.yml file: elasticsearch: image: elasticsearch:7.6.2 container_name: elasticsearch ports: - '9200:9200' -…
I'm using this conf file to overwrite the @timestamp field in ElasticSearch, but I automatically get an _dateparsefailure flag: input { jdbc { jdbc_driver_library => "C:/path/to/mariadb-java-client.jar" statement => "SELECT '${FIELD}' as field, from ${TABLE_NAME}" tracking_column => "timestamp" tracking_column_type => "timestamp"…
I have a problem about generating log info when I work in Docker. There is no issue to write logs in log file in localhost. I cannot see any new logs when I implement the CRUD process during docker. How…
I have the following docker containers docker create --name=elasticsearch --restart=always --network=infrastructure_network --network-alias=elasticsearch -e TZ=Etc/UTC -e discovery.type=single-node -e "ES_JAVA_OPTS=-Xms6g -Xmx6g" -e ELASTIC_PASSWORD="foobar" -p 9200:9200 -p 9300:9300 -v elasticsearch:/usr/share/elasticsearch elasticsearch:8.0.0 docker create --name=logstash --restart=always --network=infrastructure_network --network-alias=logstash -e TZ=Etc/UTC -p 5040:5040 -p 8514:8514/udp…
I am trying to write Logstash filter using grok logic pattern. I am logging nginx error log. My problem is the log from the nginx contains 2 field which are not present in every error logs. I have used this…
Validating logstash-output-clickhouse-0.1.0.gem Installing logstash-output-clickhouse Plugin version conflict, aborting ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-mixin-http_client": In snapshot (Gemfile.lock): logstash-mixin-http_client (= 6.0.1) In Gemfile: logstash-filter-http java was resolved to 1.0.2, which depends on logstash-mixin-http_client (>=…
I am trying to parse modsecurity audit logs which are natively being logged in JSON format. However when I am using logstash to ingest in elastic stack those appears like below and actual needed fields are not being parsed in…