Arch Linux

I opened this to the beats project as well (#12903), because it's unclear where this issue is originating.

I have elasticsearch, kibana, and packetbeat all running in Arch Linux.

When I try to setup the default packetbeat dashboard with the following command:

packetbeat -c /etc/packetbeat/packetbeat.yml \
-path.home /usr/share/packetbeat \
-path.config /etc/packetbeat \
-path.data /var/lib/packetbeat \
setup --dashboards

I see the following error:

Loading dashboards (Kibana must be running and reachable)
Exiting: Failed to import dashboard: Failed to load directory /usr/share/packetbeat/kibana/7/dashboard:
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-cassandra.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Cassandra ResponseKeyspace [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-dhcpv4.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Message Types over Time [Packetbeat DHCPv4] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-dns-overview.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'DNS Query Summary [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-dns-tunneling.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Unique FQDNs per eTLD+1 ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-flows.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Top Hosts Creating Traffic [Packetbeat Flows] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-http.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'HTTP Transactions [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-mongodb.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Navigation [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-mysql.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'MySQL Errors [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-nfs.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'NFS clients pie chart [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-overview.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'HTTP Transactions [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-pgsql.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Navigation [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-thrift.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Navigation [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}
error loading /usr/share/packetbeat/kibana/7/dashboard/Packetbeat-tls.json: returned 400 to import file: <nil>. Response: {"statusCode":400,"error":"Bad Request","message":"Failure attempting to migrate saved object 'Navigation [Packetbeat] ECS' - SyntaxError: Unexpected token o in JSON at position 1"}

These are all totally valid json files however, which I can confirm by just parsing them with Ruby:

[root@logs dashboard]# irb
irb(main):001:0> require 'json'
=> true
irb(main):002:0> JSON.parse(File.read("Packetbeat-http.json"))
=> {"objects"=>[{"attributes"=>{.....
A little googling suggests that this might be caused because unparsed JSON is expected, but rewriting the files to be unparsed JSON strings also fails with go Unmarshal errors.

Something interesting to note, is that in the arch package for metricbeat, the JSON structure is a little different. The fields:

attributes.kibanaSavedObjectMeta.searchSourceJSON
attributes.optionsJSON
attributes.panelsJSON
attributes.uiStateJSON
attributes.visState

all exist as stringified unparsed JSON, which is different than their current state on master. I don't see where that changed in the git history. I wrote a little script to make that so for the packetbeat dashboard definitions as well and I just get a different JSON error.

Using packetbeat, elasticsearch, and kibana all version 7.2.0-1.