Serverless configured lambdas running on AWS.
For initial setup, please create a muxer profile in your
~/.aws/credentials file. Serverless will use this when running the commands
listed below.
[muxer]
aws_access_key_id = <access-key-id-value>
aws_secret_access_key = <access-key-value>
region=us-east-1
output=json
Run the following commands from this directory. This will install serverless within the lambdas directory and ensure you are logged in to serverless prior to deploying changes to AWS.
$ npm installNote: The first time you login you will have to authenticate with Github in the Browser
Prior to deploying changes you will need to add secret keys to the SSM Parameter Store; This can be done via the UI or using the following commands (if you have the relevant environemntal variables configured).
This creates the keys and values in the SSM parameter store:
$ aws ssm put-parameter --name googleCalendarApiToken --region=us-east-1 --type String --value ${GOOGLE_DEVELOPER_KEY}
$ aws ssm put-parameter --name eventbriteApiToken --region=us-east-1 --type String --value ${EVENTBRITE_API_TOKEN}
$ aws ssm put-parameter --name meetupcomApiToken --region=us-east-1 --type String --value ${MEETUP_API_TOKEN}This reads the keys and values from the SSM paramter store to ensure they have been created successfully:
$ aws ssm get-parameters --region=us-east-1 --name googleCalendarApiToken eventbriteApiToken meetupcomApiToken
Lambdas used to get and process events from Meetup.com. There are helper scripts
to aid in deployment and development. Use npm run <command> with any of the
below.
Deploy (or redeploy) the lambdas and all associated infrastructure. Do this
when setting up for the first time, or whenever the serverless.yml file
changes
This lambda pulls data from the Meetup.com API and saves it to S3.
You will need to create a new entry in the AWS Systems Manager Parameter store
for the Meetup.com API token. Make a secure string with the name
meetupcomApiToken and the value of your Meetup.com API token. Serverless will
then automatically pull that value into your lambda as an environment variable.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda creates files in S3, you should see new JSON files created with the data pulled from Meetup.com.
Invoke the lambda locally. Use this for development. You will need to pass a Meetup.com API token as an environment variable:
MEETUPCOM_API_TOKEN=<token-value> npm run meetupcom:invoke-local
The local lambda will also not have permissions to write the files to S3 and you
will receive an "Access Denied" error. Instead you may want to comment out the
uploadTo calls whilst in development.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.
This lambda takes the Meetup.com JSON data which has been saved to S3, transforms it into a standardised format and saves it back to S3.
This lambda is triggered by the creation of the source file by the producer lambda.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda is triggered by the creation of files in the producer bucket, this may not run correctly. It can be invoked by calling the producer invoke command instead.
Invoke the lambda locally. Use this for development.
The local lambda will need to be provided with an event object which contains mock values for the newly created file. Samples of the AWS event objects can be found at: https://docs.aws.amazon.com/lambda/latest/dg/eventsources.html#eventsources-s3-put
The local lambda will not have permissions to read / write files to / from S3
and you will receive an "Access Denied" error. Instead you may want to comment
out the getListFromS3, getFromS3 and uploadTo calls whilst in development.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.
Lambdas used to get and process events from Eventbrite. There are helper scripts
to aid in deployment and development. Use npm run <command> with any of the
below.
Deploy (or redeploy) the lambdas and all associated infrastructure. Do this
when setting up for the first time, or whenever the serverless.yml file
changes
This lambda pulls data from the Eventbrite.com API and saves it to S3.
You will need to create a new entry in the AWS Systems Manager Parameter store
for the Eventbrite API token. Make a secure string with the name
eventbriteApiToken and the value of your Eventbrite API token. Serverless will
then automatically pull that value into your lambda as an environment variable.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda creates files in S3, you should see new JSON files created with the data pulled from Eventbrite.
Invoke the lambda locally. Use this for development. You will need to pass an Eventbrite API token as an environment variable:
EVENTBRITE_API_TOKEN=<token-value> npm run eventbrite:invoke-local
The local lambda will also not have permissions to write the files to S3 and you
will receive an "Access Denied" error. Instead you may want to comment out the
uploadTo calls whilst in development.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.
This lambda takes the Eventbrite JSON data which has been saved to S3, transforms it into a standardised format and saves it back to S3.
This lambda is triggered by the creation of the source file by the producer lambda.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda is triggered by the creation of files in the producer bucket, this may not run correctly. It can be invoked by calling the producer invoke command instead.
Invoke the lambda locally. Use this for development.
The local lambda will need to be provided with an event object which contains mock values for the newly created file. Samples of the AWS event objects can be found at: https://docs.aws.amazon.com/lambda/latest/dg/eventsources.html#eventsources-s3-put
The local lambda will not have permissions to read / write files to / from S3
and you will receive an "Access Denied" error. Instead you may want to comment
out the getFromS3 and uploadTo calls whilst in development.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.
Lambdas used to get and process events from the Farset Labs calendar. There are
helper scripts to aid in deployment and development. Use npm run <command>
with any of the below.
Deploy (or redeploy) the lambdas and all associated infrastructure. Do this
when setting up for the first time, or whenever the serverless.yml file
changes
This lambda pulls data from the Farset Labs Google calendar JSON endpoint and saves it to S3.
You will need to create a new entry in the AWS Systems Manager Parameter store
for the Google Calendar API token. Make a secure string with the name
googleCalendarApiToken and the value of your Eventbrite API token. Serverless
will then automatically pull that value into your lambda as an environment
variable.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda creates files in S3, you should see new JSON files created with the data pulled from the Farset Labs calendar.
Invoke the lambda locally. Use this for development. You will need to pass an Google Calendar API token as an environment variable:
GOOGLE_CALENDAR_API_TOKEN=<token-value> npm run farsetlabs:producer:invoke-local
The local lambda will not have permissions to write the files to S3 and you
will receive an "Access Denied" error. Instead you may want to comment out the
uploadTo calls whilst in development.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.
This lambda takes the Farset Labs Google calendar JSON data which has been saved to S3, transforms it into a standardised format and saves it back to S3.
This lambda is triggered by the creation of the source file by the producer lambda.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda is triggered by the creation of files in the producer bucket, this may not run correctly. It can be invoked by calling the producer invoke command instead.
Invoke the lambda locally. Use this for development.
The local lambda will need to be provided with an event object which contains mock values for the newly created file. Samples of the AWS event objects can be found at: https://docs.aws.amazon.com/lambda/latest/dg/eventsources.html#eventsources-s3-put
The local lambda will not have permissions to read / write files to / from S3
and you will receive an "Access Denied" error. Instead you may want to comment
out the getFromS3 and uploadTo calls whilst in development.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.
Lambdas used to push events to the Muxer service. There are helper scripts to
aid in deployment and development. Use npm run <command> with any of the
below.
Deploy (or redeploy) the lambdas and all associated infrastructure. Do this
when setting up for the first time, or whenever the serverless.yml file
changes
This lambda takes the source agnostic standardised JSON data for events, which has been saved to S3, and publishes it to the Muxer events service.
This lambda is triggered by the creation of the source file by the transformer lambdas.
Update just the handler functionality. Do then whenever you change the functionality of the lambda.
Invoke the lambda on AWS. As this lambda is triggered by the creation of files in the transformed bucket, this may not run correctly. It can be invoked by calling the producer/transformer invoke commands instead, to push new data into the bucket.
Invoke the lambda locally. Use this for development.
The local lambda will need to be provided with an event object which contains mock values for the newly created file. Samples of the AWS event objects can be found at: https://docs.aws.amazon.com/lambda/latest/dg/eventsources.html#eventsources-s3-put
The local lambda will not have permissions to read files from S3 and you will
receive an "Access Denied" error. Instead you may want to comment out the
getFromS3 call whilst in development.
In addition, in development you will not want to push events directly to the production Muxer events service. This should be pointed to your locally running service instead.
Pulls the logs from cloudwatch of the last lambda run. Useful for debugging.