How to insert data using docker compose

AbdelrahmanElayashy
June 13, 2022
278 views
0 votes
2 Answers

Overview: I would like to insert data in file.json in db within docker-compose using CURL.

My Try: I have tried to mount file.json inside the FROST container, then run the command CURL within docker-compose.

curl -X POST -H "Content-Type: application/json" -d file.json http://localhost:8080/FROST-Server/v1.1/Things

as the result the docker-compose yaml is so

version: '2'

services:
  web:
    image: fraunhoferiosb/frost-server:latest
    environment:
      - serviceRootUrl=http://localhost:8080/FROST-Server
      - http_cors_enable=true
      - http_cors_allowed.origins=*
      - persistence_db_driver=org.postgresql.Driver
      - persistence_db_url=jdbc:postgresql://database:5432/sensorthings
      - persistence_db_username=sensorthings
      - persistence_db_password=ChangeMe
      - persistence_autoUpdateDatabase=true
    ports:
      - 8080:8080
      - 1883:1883
    volumes:
      - ./file.json:file.json
    depends_on:
      - database
    command:
      - curl
      - -X
      - POST
      - --data-binary
      - ./file.json
      - -H
      - "Content-Type: application/json" # <-- YAML string quoting
      - -H
      - "Accept: application/json"
      - "http://localhost:8080/FROST-Server/v1.1/Things"
  database:
    image: postgis/postgis:14-3.2-alpine
    environment:
      - POSTGRES_DB=sensorthings
      - POSTGRES_USER=sensorthings
      - POSTGRES_PASSWORD=ChangeMe
    volumes:
      - postgis_volume:/var/lib/postgresql/data
volumes:
  postgis_volume:

After running docker-compuse up, i got this error

Creating network "frost_default" with the default driver
Creating frost_database_1 ... done
Creating frost_web_1      ... done
Attaching to frost_database_1, frost_web_1
database_1  |
database_1  | PostgreSQL Database directory appears to contain a database; Skipping initialization
database_1  |
database_1  | 2022-06-13 11:24:06.420 UTC [1] LOG:  starting PostgreSQL 14.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027, 64-bit
database_1  | 2022-06-13 11:24:06.421 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
database_1  | 2022-06-13 11:24:06.421 UTC [1] LOG:  listening on IPv6 address "::", port 5432
database_1  | 2022-06-13 11:24:06.458 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
database_1  | 2022-06-13 11:24:06.486 UTC [22] LOG:  database system was shut down at 2022-06-13 09:58:59 UTC
database_1  | 2022-06-13 11:24:06.498 UTC [1] LOG:  database system is ready to accept connections
web_1       |   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
web_1       |                                  Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
web_1       | curl: (7) Failed to connect to localhost port 8080: Connection refused
frost_web_1 exited with code 7

note: docker-compose.yml and file.json are in same path.

file.json

{
    "name": "Car",
    "description": "Smart connected car equipped with sensors.",
    "properties": {
        "vin": "5YJ3EDEB5KF223462"
    },
    "Locations": [
        {
            "name": "Parking lot",
            "description": "The parking lot of the fictive company.",
            "encodingType": "application/vnd.geo+json",
            "location": {
                "type": "Point",
                "coordinates": [8.419432640075684, 49.01395040719586]
            }
        }
    ],
    "Datastreams": [
        {
            "name": "Oil pump",
            "description": "Measuring the motor oil pressure.",
            "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
            "unitOfMeasurement": {
                "name": "Bar",
                "symbol": "bar",
                "definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#Bar"
            },
            "Sensor": {
                "name": "OIL_PRES_SENSOR2",
                "description": "Oil pressure sensor",
                "encodingType": "application/pdf",
                "metadata": "..."
            },
            "ObservedProperty": {
                "name": "Pressure",
                "definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#Pressure",
                "description": "The oil pressure."
            },
            "Observations": [
                {
                    "phenomenonTime": "2022-01-10T10:00:00Z",
                    "result": 2.1
                },
                {
                    "phenomenonTime": "2022-01-10T10:01:10Z",
                    "result": 2.3
                },
                {
                    "phenomenonTime": "2022-01-10T10:02:20Z",
                    "result": 2.7
                },
                {
                    "phenomenonTime": "2022-01-10T10:03:30Z",
                    "result": 2.9
                },
                {
                    "phenomenonTime": "2022-01-10T10:04:40Z",
                    "result": 4.1
                },
                {
                    "phenomenonTime": "2022-01-10T10:05:50Z",
                    "result": 3.7
                }
            ]
        },
        {
            "name": "Motor Temperature",
            "description": "The temperature of the motor.",
            "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
            "unitOfMeasurement": {
                "name": "Centigrade",
                "symbol": "C",
                "definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#DegreeCentigrade"
            },
            "Sensor": {
                "name": "DHT22/Temperature",
                "description": "Temperature sensor of a DHT22",
                "encodingType": "application/pdf",
                "metadata": "https://www.sparkfun.com/datasheets/Sensors/Temperature/DHT22.pdf"
            },
            "ObservedProperty": {
                "name": "Temperature",
                "definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#ThermodynamicTemperature",
                "description": "The temperature."
            },
            "Observations": [
                {
                    "phenomenonTime": "2019-03-14T10:00:00Z",
                    "result": 21.0
                },
                {
                    "phenomenonTime": "2019-03-14T10:01:00Z",
                    "result": 23.1
                },
                {
                    "phenomenonTime": "2019-03-14T10:02:00Z",
                    "result": 40.5
                },
                {
                    "phenomenonTime": "2019-03-14T10:03:00Z",
                    "result": 47.1
                },
                {
                    "phenomenonTime": "2019-03-14T10:04:00Z",
                    "result": 32.2
                },
                {
                    "phenomenonTime": "2019-03-14T10:05:00Z",
                    "result": 30.3
                }
            ]
        },
        {
            "name": "Access",
            "description": "The access state (e.g., open, closed) of the vehicle.",
            "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
            "unitOfMeasurement": {
                "name": "Centimeter",
                "symbol": "cm",
                "definition": "http://www.qudt.org/qudt/owl/1.0.0/unit/Instances.html#Centimeter"
            },
            "Sensor": {
                "name": "Distance Sensor",
                "description": "Measure the distance.",
                "encodingType": "application/pdf",
                "metadata": "..."
            },
            "ObservedProperty": {
                "name": "Length",
                "definition": "http://www.qudt.org/qudt/owl/1.0.0/quantity/Instances.html#Length",
                "description": "The length of the measured distance."
            },
            "Observations": [
                {
                    "phenomenonTime": "2019-03-14T10:00:00Z",
                    "result": 0.0
                },
                {
                    "phenomenonTime": "2019-03-14T10:01:00Z",
                    "result": 0.0
                },
                {
                    "phenomenonTime": "2019-03-14T10:02:00Z",
                    "result": 21.4
                },
                {
                    "phenomenonTime": "2019-03-14T10:03:00Z",
                    "result": 21.7
                },
                {
                    "phenomenonTime": "2019-03-14T10:04:00Z",
                    "result": 20.9
                },
                {
                    "phenomenonTime": "2019-03-14T10:05:00Z",
                    "result": 0.0
                }
            ]
        }
    ]
}

any other solution is welcome too:)

Answers

Chosen as BEST ANSWER

The End Solution of the problem. I would like to thank @RoohAllahGodazgar for helping me out.

Explanation: we need to have another image as curl image which we can run curl request on it after the FROST server is run. on curl service, we should mount first the dummy data inside the container and after that run sleep 20s so that we make sure that FROST server is run, then run crul method. after that the container will be exited.

version: '2'

services:
  curl:
    image: alpine/curl:latest
    command: sh -c "sleep 20 && curl -X POST -d @/file.json http://web:8080/FROST-Server/v1.1/Things"
    volumes:
      - ./file.json:/file.json
    depends_on:
      - web
  web:
    image: fraunhoferiosb/frost-server:latest
    environment:
      - serviceRootUrl=http://0.0.0.0:8080/FROST-Server
      - http_cors_enable=true
      - http_cors_allowed.origins=*
      - persistence_db_driver=org.postgresql.Driver
      - persistence_db_url=jdbc:postgresql://database:5432/sensorthings
      - persistence_db_username=sensorthings
      - persistence_db_password=ChangeMe
      - persistence_autoUpdateDatabase=true
    ports:
      - 8080:8080
      - 1883:1883
    depends_on:
      - database

  database:
    image: postgis/postgis:14-3.2-alpine
    environment:
      - POSTGRES_DB=sensorthings
      - POSTGRES_USER=sensorthings
      - POSTGRES_PASSWORD=ChangeMe
    volumes:
      - postgis_volume:/var/lib/postgresql/data
volumes:
  postgis_volume:

(Edit)

I think you’re executing your "curl" command before your "web" container starts. If I were you, I will run the "web" on "0.0.0.0" instead of "localhost", and use separate container to sleep for some time, then execute the curl command (don’t forget about the container entrypopoint and override it if required).

I did run your docker-compose and execute the curl manually, but I got 400:

curl -vvv -X POST --data-binary ./file.json -H "Content-Type: application/json" -H "Accept: application/json" "http://localhost:8080/FROST-Server/v1.1/Things"

Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1:8080...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /FROST-Server/v1.1/Things HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.68.0
> Content-Type: application/json
> Accept: application/json
> Content-Length: 11
> 
* upload completely sent off: 11 out of 11 bytes
* Mark bundle as not supporting multiuse
< HTTP/1.1 400 
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: Location
< Content-Length: 224
< Date: Tue, 14 Jun 2022 12:43:06 GMT
< Connection: close
< 
* Closing connection 0
{"code":400,"type":"error","message":"Unexpected character ('.' (code 46)): expected a valid value (JSON String, Number, Array, Object or token 'null', 'true' or 'false')n at [Source: (BufferedReader); line: 1, column: 2]"}

Since I’m not familiar with this service, I don’t know how to fix this, but you can test bellow docker-compose if you could fix this:

version: '2'

services:
  curl:
    image: alpine/curl:latest
    entrypoint: /bin/sh
    command:
      - -c
      - sleep 10
      - curl
      - -vvv
      - -X
      - POST
      - --data-binary
      - /file.json
      - -H
      - "Content-Type: application/json" # <-- YAML string quoting
      - -H
      - "Accept: application/json"
      - "http://web:8080/FROST-Server/v1.1/Things"
    volumes:
      - ./file.json:/file.json
    depends_on:
      - web
  web:
    image: fraunhoferiosb/frost-server:latest
    environment:
      - serviceRootUrl=http://0.0.0.0:8080/FROST-Server
      - http_cors_enable=true
      - http_cors_allowed.origins=*
      - persistence_db_driver=org.postgresql.Driver
      - persistence_db_url=jdbc:postgresql://database:5432/sensorthings
      - persistence_db_username=sensorthings
      - persistence_db_password=ChangeMe
      - persistence_autoUpdateDatabase=true
    ports:
      - 8080:8080
      - 1883:1883
#    volumes:
#      - ./file.json:/usr/local/tomcat/file.json
    depends_on:
      - database
#    command:
#      - curl
#      - -X
#      - POST
#      - --data-binary
#      - ./file.json
#      - -H
#      - "Content-Type: application/json" # <-- YAML string quoting
#      - -H
#      - "Accept: application/json"
#      - "http://localhost:8080/FROST-Server/v1.1/Things"
  database:
    image: postgis/postgis:14-3.2-alpine
    environment:
      - POSTGRES_DB=sensorthings
      - POSTGRES_USER=sensorthings
      - POSTGRES_PASSWORD=ChangeMe
    volumes:
      - postgis_volume:/var/lib/postgresql/data
volumes:
  postgis_volume:

Edit 1: volume was missing from "curl" container

Please signup or login to give your own answer.

Click here to cancel reply.