AWS Lambdas with Docker images: do I have to create a different docker image per lambda?

First, your application Docker image would not be stored inside of the Lambda. The Docker image would be stored in AWS ECR, which is the Container Registry that AWS provides for its customers. You would build your image, tag your image and then publish your image to an ECR repository that you create. The image in this ECR repository can be utilized by any AWS Service that accepts a Docker image, whether it is Lambda, ECS, EKS, Batch, etc. It is not something specific to Lambda, in other words.

Second, I would create an ECR repo per application. I would not think of it as a 1:1 between lambda and ecr, but rather a 1:1 between application and ecr repo. Think of the ECR Repo as the container for a given application. So each repo would have a Dockerfile, which uses the From instruction as so:

FROM public.ecr.aws/lambda/python:3.8 
COPY requirements.txt . 
RUN pip3 install -r requirements.txt --target "${LAMBDA_TASK_ROOT}" 
COPY app.py ${LAMBDA_TASK_ROOT}

So this app.py might be MyApp1. and it can be importing a bunch of other files that comprise of MyApp1. This corresponds to an ECR Repo you can call my-app-1. Then you will have a second application, where the app.py and imported files are vastly different and so you would have a second ecr repo to hold the layers of that application.

Then for your Lambda function parameters, you will specify the image uri, which refers to the ECR URI. You will specify your package type as "Image". Here is a rudimentary example in Terraform Infrastructure As Code to illustrate the point:

resource "aws_lambda_function" "function" {
  function_name     = "${var.name_prefix}-my-lambda${var.name_suffix}"
  description       = "My Lambda Function"
  image_uri         = var.image_uri
  package_type      = "Image"
  timeout           = var.timeout
  memory_size       = var.memory_size
  role              = var.role_arn
  tags              = var.tags
} 

The image_uri variable would come from the ECR Repo that was created. So the variable would look something like this:

resource "aws_ecr_repository" "repo" {
  name = "Your Repo Name"
}

resource "null_resource" "ecr_image" {
  triggers = {
    docker_file       = md5(file("${path.module}/../../Dockerfile"))
    app_file          = md5(file("${path.module}/../../app.py"))
  }

  provisioner "local-exec" {
    command = <<EOF
      aws ecr get-login-password --region ${var.region} | docker login --username AWS --password-stdin ${local.account_id}.dkr.ecr.${var.region}.amazonaws.com
      cd ${path.module}/../../
      docker build -f Dockerfile -t ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag} .
      docker push ${aws_ecr_repository.repo.repository_url}:${local.ecr_image_tag}
    EOF
  }
}

data "aws_ecr_image" "lambda_image" {
  depends_on = [
    "null_resource.ecr_image"
  ]

  repository_name = "Your Repo Name"
  image_tag       = local.ecr_image_tag
}


output "image_uri" {
  value = aws_ecr_repository.repo.repository_url}@${data.aws_ecr_image.lambda_image.id
}

In the above example, I am using Terraform, but you could just as easily reproduce the same scenario in CloudFormation or directly through the AWS CLI.