Let's say I have a message producer, which sends a very high number of messages with versions V1, V2, ..., VN, in a short timespan. We have a service, which is not required to react to every such message. We…
I've been trying to ingest data from an AWS S3 bucket to AWS OpenSearch Serverless using AWS Lambda but I can't find any documentation on how to do it, any ideas? I've seen plenty of AWS OpenSearch Service examples but…
I have the requirements to create an IAM user programmatically in AWS. Unfortunately all of the examples I have found online require the nuget package AWSSDK. The AWSSDK package is deprecated and I do not want to use something that…
Using PySpark 2.4.7 My goal is to write PySpark DataFrame into specific number of parquet files into AWS S3. Say I want to write PySpark DataFrame into 10 parquet files. This is my approach df.repartition(10).write.mode("append").partitionBy(partition_cols).saveAsTable(<db.table>) This writes 10 parquet files…
I'm trying to run a simple code example to train and predict a ML algorithm using the Opensearch ML Commons plugin on a AWS managed Opensearch cluster. The cluster is on OS version 2.5, which definitely supports the plugin. The…
I would like to retrieve the DB Option Group (OptionGroupName) for an RDS by using the aws rds describe-db-instances --query and --filters. Just FYI I can not use jq to filter the json response from aws rds describe-db-instances.
I am unable to understand what mistake I am doing. Please look at this and help. I have chosen Linux family operating system in task definiton. Getting this error. Please take a look into below screenshots. This is the cloud…
I'm starting with Amazon EC2 usage for some ML training on a GPU. What I want to do after I launch and ssh to my EC2 (Spot) instance: clone my GitHub repository (first setup access) download dataset prepare dataset (unzip,…
What is the exact event type from the @types/aws-lambda package for a Lambda function hosted using function URL?
I have created a Glue catalog in my account. It has 1 DB and 1 table. I followed this guide from AWS and created my EMR cluster. However, when I run spark-shell and try to access Glue catalog, I am…