I have a stored procedure in redshift and calling it by lambda function …problem is with max time limit that is 15min for lambda function or that it give timeout error but our stored procedure require time more than that time to execute completly….so how to handle this issue. ..or any other way to call stored procedure in which no time limit will be there.
3
Answers
It’s difficult to say what exactly will work since your implementation and trigger details are not clear, but you would likely be able to to do this with AWS Step Functions instead of Lambda.
This blog post outlines a sample implementation using Step Functions and Redshift Data API – https://aws.amazon.com/blogs/big-data/etl-orchestration-using-the-amazon-redshift-data-api-and-aws-step-functions-with-aws-sdk-integration/
There’s no way you can bypass the time limit for the AWS lambda, you can use AWS Step Function to iterate multiple lambda calls, which it’s very typical way to handle the tons of loads of lambda, but you will have a way to keep the state for the lambda calls
Since you simply wish to start execution of the Stored Procedure, you could make a call to the Redshift API service (which is different to a normal Redshift call).
You could call
execute_statement()with an SQL statement to start running the store procedure. The Lambda function does not need to maintain the connection – it will continue to execute. Therefore, the 15-minutes Lambda limit is not relevant.