Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – Duplicity SSLEOFError: EOF occurred in violation of protocol

I have duplicity running for years, stable and without problems. Now I encountered the following problem when I backup my files to Amazon S3.

All files in the format "duplicity-full.YYYYMMDD1ThhmmssZ.volX.difftar.gpg" are written to Amazon S3 successfully. At least I can see them in the directory/bucket.

Unfortunately, when uploading the "duplicity-full-signatures.YYYYMMDD1ThhmmssZ.sigtar.gpg" an error occurs stating:

Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 387, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 574, in move
    self.__do_put(source_path, remote_filename)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 544, in __do_put
    self.backend._put(source_path, remote_filename)
  File "/usr/lib/python3/dist-packages/duplicity/backends/_boto_single.py", line 265, in _put
    self.upload(source_path.name, key, headers)
  File "/usr/lib/python3/dist-packages/duplicity/backends/_boto_single.py", line 319, in upload
    key.set_contents_from_filename(filename, headers,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 1375, in set_contents_from_filename
    return self.set_contents_from_file(fp, headers, replace, cb,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 1307, in set_contents_from_file
    self.send_file(fp, headers=headers, cb=cb, num_cb=num_cb,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 760, in send_file
    self._send_file_internal(fp, headers=headers, cb=cb, num_cb=num_cb,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 957, in _send_file_internal
    resp = self.bucket.connection.make_request(
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/connection.py", line 667, in make_request
    return super(S3Connection, self).make_request(
  File "/usr/local/lib/python3.10/dist-packages/boto/connection.py", line 1070, in make_request
    return self._mexe(http_request, sender, override_num_retries,
  File "/usr/local/lib/python3.10/dist-packages/boto/connection.py", line 1030, in _mexe
    raise ex
  File "/usr/local/lib/python3.10/dist-packages/boto/connection.py", line 939, in _mexe
    response = sender(connection, request.method, request.path,
  File "/usr/local/lib/python3.10/dist-packages/boto/s3/key.py", line 856, in sender
    http_conn.send(chunk)
  File "/usr/lib/python3.10/http/client.py", line 999, in send
    self.sock.sendall(data)
  File "/usr/lib/python3.10/ssl.py", line 1266, in sendall
    v = self.send(byte_view[count:])
  File "/usr/lib/python3.10/ssl.py", line 1235, in send
    return self._sslobj.write(data)
 ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2426)

Attempt of move Nr. 1 failed. SSLEOFError: EOF occurred in violation of protocol (_ssl.c:2426)

After 5 attemps, the upload is stopped. Checking the upload online monitor of my router I see that a file is uploaded, but only for short time (like 10 sec). In addition, I cannot find the "duplicity-full-signatures.YYYYMMDD1ThhmmssZ.sigtar.gpg" in the S3 directory/bucket.
This problem only occurs with one big directory (~1 TB). Other directories work without any problems (same user, same password, same keys etc.)

I have Ubuntu 22.04.4 LTS with kernel 5.15.0-116-generic with duplicity 0.8.21.

Any ideas? How to debug? And how to find the problem? I got the error message running

duplicity --verbosity d
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    When trying to run with boto3+s3:// I get:

    Temp has 41315815424 available, backup will use approx 272629760.
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 387, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 607, in list
    return [tobytes(x) for x in self.backend._list()]
  File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line 167, in _list
    self.reset_connection()
  File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line 95, in reset_connection
    self.s3.meta.client.head_bucket(Bucket=self.bucket_name)
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 565, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 1017, in _make_api_call
    raise error_class(parsed_response, operation_name)
 botocore.exceptions.ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden

Attempt of list Nr. 1 failed. ClientError: An error occurred (403) when calling the HeadBucket operation: Forbidden
Backtrace of previous error: Traceback (innermost last):
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 387, in inner_retry
    return fn(self, *args)
  File "/usr/lib/python3/dist-packages/duplicity/backend.py", line 607, in list
    return [tobytes(x) for x in self.backend._list()]
  File "/usr/lib/python3/dist-packages/duplicity/backends/s3_boto3_backend.py", line 170, in _list
    for obj in self.bucket.objects.filter(Prefix=self.key_prefix):
 AttributeError: 'NoneType' object has no attribute 'objects'

Attempt of list Nr. 2 failed. AttributeError: 'NoneType' object has no attribute 'objects'

  2. 0

    sounds like you hit the upper file size limit of Amazon S3 for single put operations (5GB). unfortunately the manifest is not split to volume sizes even in recent duplicity.

    but, as Amazon S3 documentation states files (objects) up to 5TB can be uploaded with multipart upload API, which is the default in current duplicity 3.x as it uses the current boto3 python module.

    your old duplicity already had this backend integrated, albeit only as alternative only. you can use it using the boto3+s3:// target url scheme. you will need to have package python3-boto3 installed.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search