Content translation system using Contentstack Webhook, AWS Lambda, XTM Human Translation

XTM is a cloud-based content translation management solution. With its integrated CAT tool, it helps organizations to streamline complex localization processes and maximize translation reuse.

Its scalable and flexible architecture allows it to integrate with apps such as Contentstack seamlessly. In this guide, we will go through the steps required to set up a language translation system for your Contentstack-powered website by using Contentstack Webhooks, AWS Lambda, and XTM human translation.

Process overview: We will first set up the essentials, such as creating a stack and adding a few languages (locales), in Contentstack. We will then create a workflow and add four stages to it. We will add a template in XTM, configure it for human translation, and then set up two lambda functions to help us create the bridge between Contenstack and XTM for translation.

We'll also set up a webhook such that when the workflow stage in an entry changes (let's say from Draft to Send for Translation), the webhook will issue a trigger to our first lambda function. It will initiate the translation process with XTM by creating a JSON file of our entry and uploading it to the XTM project.

We will then manually get the file translated and once it is done, the second lambda function will be invoked. I will update the entry in the required language along with changing its workflow stage to “Review.”

Prerequisites

AWS account and access to AWS Lambda, AWS API Gateway, and AWS IAM
Contentstack account
XTM Project Manager account
XTM Linguist account
Working knowledge of AWS Lambda and API Gateways

Note: For this example, we assume that you already have a Project Manager and a Linguist account in XTM that will help you create projects and translations in XTM. If not, please sign up and get your account created. You will receive all the details required to log in to XTM such as URL, username, password, and so on.

Steps for Execution

Here are the steps required to set up this translation system:

Create a project template in XTM
Set up the essentials in Contentstack
Set up the workflow for translation in Contentstack
Set up the AWS lambda functions for translation
Trigger a webhook to initiate translation
Try out the steps

Let's get started!

Create a Project Template in XTM
For this exercise, you need a project manager (or an admin) account with XTM so that you can create templates in the XTM app. You can sign up with one of their plans and get started with setting up your account.
Assuming that you have a project manager account in XTM and have logged in the app, the first thing that you need to do is to create a template in XTM. To do this, follow the steps given below:
1. Once you log in to the XTM app, you will be on the Projects page (under Project list). Click on Add project as shown below:
2. On the page that opens, under the General information section, the Customer name is auto-populated. You can provide a new customer if you want and leave the remaining options blank.
3. Under the Translation section, provide the Source language (English USA in our example) and Target languages [Chinese (simplified), French (France) in our example]. You can add more languages if you like and leave the other options unchanged.
4. Inside the Workflow section, you can set up a workflow by selecting different options from the Main workflow dropdown. For our example, we have selected Translate, Review as the stages of our XTM workflow:
5. Leave the options inside the Settings section to their default values and inside the Machine translation section, uncheck the Use Amazon machine translation option, if you will be manually translating the content.
  
  Leave other options to their default values and click on the Save as template button.
6. The Create template modal will open. Provide a Name to your template and an optional Description. You can leave the default Customer specific option selected for Type as shown below:
7. Click on the Save button and you will get a successful message upon creation. Click on Close to close the Create template modal.
8. Your template is now ready. Go to the Templates option next to (Project list) and you will see the template that you just created.
9. Now hover over the "i" icon to the right corresponding to your template and you should be able to view the template ID. Make a note of it.
The XTM template is now ready. Similarly, you can create multiple templates for your project. We have created a couple of templates for the purpose of this exercise. We will require the template IDs while creating a content type in Contentstack.
Let's move ahead and set up the essentials in Contentstack.
Set up the Essentials in Contentstack
To set up the essentials, log in to your Contentstack account and follow the steps given below:
1. Create a stack, add a content type (for our exercise, we have created a Single Content Type named XTM). In our example, we have used “title,” “url,” “single_line (named Heading),” “rich_text_editor (named Body)”, and Select fields. We will translate the content of these fields. Content of special fields such as File, Boolean, Reference, Select will not be translated.
  
  The Select field that we have added (named XTM Project Templates in the above screenshot) is where you will add the template IDs you created in the above step. Ensure that the ID of this select field is the same shown in the screenshot below (xtm_project_templates).
  
  If you are using our code example, ensure that the format of adding the IDs is similar to the one shown below:
  
  Note: You can add Template-33928, Template-30684, and so on so that it's easier for you to identify and select the required template in the entry. We will use the ID separated by a hyphen for identifying them in the entry page.
2. Create a management token for your stack which we will use later while setting up the lambda function.
3. Next, create languages (locales) in your content type (XTM). In our example, we have added a few languages:
  
  English [en-us] which is the source and our master language
  Chinese - China [zh-cn] is another language
  French - France [fr-fr] which will be our target language
  
  Note: Ensure that the languages you add are supported by XTM.
With these steps, we have set up the essentials in Contentstack. Let's now move ahead with creating a workflow.
Set up the Workflow for Translation in Contentstack
To set up workflows for the translation process, perform the following steps:
1. Hover over the Settings gear icon and click on Workflows.
2. On the Workflow Settings page, click on + ADD WORKFLOW.
3. Provide a suitable name to your workflow and an optional description.
4. Under the Scope option, select if this workflow should be applied to All Content Types or Specific Content Type(s). For our example, we will select the Specific Content Types(s) option and then select our content type, XTM, and then click on Add as shown below:
5. Inside the WORKFLOW STAGES option, add four stages (for example, Draft, Send for Translation, Review, and Completed).
  
  Additional resource: Learn more about workflow and its stages, refer to the set up workflows guide.
6. Lastly, click on the Enable Workflow checkbox and then on the Save button.
With these steps, our workflow is ready. When the editor changes the workflow stage from “Draft” to “Send for Translation,” the webhook will be triggered and call the first lambda function (we will set these up later in the guide).
Now go back to the Workflow Settings page and note down the WORKFLOW ID as shown below (we will need this ID later when we setup our lambda function):

With these steps, we have created our workflow. Let's now move ahead and set up lambda functions.
Set up the AWS Lambda Functions
For this exercise, we will create two lambda functions.
1. The first lambda function (once invoked through our webhook) will take the content of the entry and upload it as a JSON file in the XTM app.
2. The second lambda function (once invoked through the XTM webhook after the file is translated) will take the translated file and upload it in the entry in the targeted locale in Contentstack and update the entry workflow state to Review.
Set up the First Lambda Function
Let's now create the first lambda function by using the steps given below:
1. Log in to your AWS Management Console, select Lambda from the Services list.
2. Click on the Create function button and then the Author from Scratch option.
3. Provide a name to your lambda function inside the Function name field, select Node.js 14.x as your Runtime language, and click on the Create function button.
4. You will get a success message on creation of the lambda function. For this exercise, we have created the sample code for both lambda functions. If Extensions are part of your plan, contact our Support team to get the code for the extension.
5. Once you get the code, open command prompt and move inside the folder named before-translation.
  1. Then, install the required dependencies by running the following command:
```
npm install
```
  2. Now, run the following command to create a zip file:
```
npm run build
```
    After running the above command, you'll get a before-translation.zip folder. This is the zipped file that we will use in our lambda function.
    
    Similarly, in your command prompt, move inside the folder named after-translation, install the required dependencies, and build it to create a zip folder named after-translation.zip. This folder will be used for the second lambda function.
    
    Note: The npm build command we discussed above will work for Mac and Linux users. If you are on Windows, the npm build command may not work. In that case, after installing the dependencies, you will have to manually zip the code files and name them before-translation.zip and after-translation.zip.
6. Now in your AWS console, inside the Code source section, click on the Upload from dropdown and select .zip file.
7. In the Upload a .zip file modal, click on the Upload button, and select the file named before-translation.zip. Then, click on Save.
8. In the Runtime settings option, keep Handler as index.handler.
9. Scroll up and select the Configuration tab. Click on the Environment variable option on the left and add the following variables inside it by clicking on Edit and then Add environment variable. Once added, click on Save.
  
  api_key: <<Stack API Key>>
  base_url: https://api.contentstack.io/v3... (for EU: https://eu-api.contentstack.com/v3/; for Azure NA: https://azure-na-api.contentstack.com/v3/)
  customer_id: 23
  job_finished_callback: <<API endpoint of the second lambda function (we'll set it up next)>>
  management_token: <<Stack Management Token>>
  master_locale: en-us
  user_company_name: <<XTM User company that you created while setting up the account>>
  user_identifier: 20
  xtm_password: <<Your XTM account password>>
  xtm_base_url: https://api-test.xtm-intl.com/...
  
  Note: To get the customer_id, click on the Customers tab at the top (next to Projects) in XTM. Then, hover over the "i" icon at the right to get it to get the ID of the respective customer. Similarly, to get the user_identifier key, click on the Users tab at the top and then hover over the "i" icon and note down the ID.
10. Your first lambda function is now ready. Let's add a trigger (API Gateway) for this lambda function. To do this, scroll up to the Function overview section and click on + Add trigger.
11. On the Add trigger screen, from the Select a trigger dropdown, select API Gateway.
12. From the API dropdown, select Create an API. Then, select REST API inside the API type block, select Open from the Security dropdown, and click on Add.
13. An API for your lambda function is now created. Inside the Triggers section, you will see the Details link. Click on it and you will find your API endpoint. Make a note of it as we will need it while setting up our webhook in Contentstack.
We are now ready with our lambda function, let's move ahead and set up the second lambda function.
Set up the Second Lambda Function
Let's set up the second lambda function which will be invoked once the translation gets completed. Follow the same steps, as mentioned above. But this time, upload the after-translation.zip file in your lambda function.
Also, use the following environment variables while setting up the function:

api_key: <<Stack's API key>>
management_token: <<Stack's Management Token>>
base_url: https://api.contentstack.io/v3... (for EU: https://eu-api.contentstack.com/v3/; for Azure NA: https://azure-na-api.contentstack.com/v3/)
workflow_uid: <<Workflow UID that we created above>>
user_company_name: <<XTM account company name>>
user_identifier: 20
xtm_password: <<XTM account password>>
xtm_base_url: https://api-test.xtm-intl.com/project-manager-api-rest/
With these steps the second lambda function is now ready. Now, follow similar steps to create an API trigger for this lambda function as well and use its API endpoint as the value of the callbackUrl environment variable in your first lambda function.
Let's now move ahead and create a webhook that will invoke the first lambda function once the workflow stage changes to Send for Translation.
Trigger a Webhook to Initiate Translation
To create and set up a webhook in Contentstack, log in to your Contentstack account and perform the following steps:
1. Hover over the Settings gear icon, click on Webhooks, and on the Webhooks page, click on + New Webhook.
2. On the Create Webhook page, fill up the Name field (for example, XTM Translation).
3. In the URL to notify field, enter the URL (the API endpoint of the first lambda function) that you generated in the previous step.
4. Scroll down to the When section for creating a trigger for the webhook as shown below:
5. Ensure to check the Enable Webhook option and click on the Save button to save your settings.
With these steps, we have completed the entire set up and now we are ready to try it out.
Try out the Steps
We are now ready to test the setup.
1. Create an entry inside the content type (in the en-us locale). Select the XTM template and after you have added the details in other fields, change the workflow stage by selecting Send for Translation from the WORKFLOW DETAILS section on the right side.
  
  Note: You can only send content from your master locale to XTM for translation. If you try sending the data from other locales, it will not be translated and the webhook will throw an error.
2. This will trigger the webhook and it will invoke the first lambda function. The lambda function will generate the JSON file of the entry and place it in the XTM account.
3. Now log back in to your XTM account (if you have logged out) and click on the Project list tab at the top (under the Projects main tab).
4. A project will get created automatically with the name of your content type, entry UID, and XTM template name as shown below:
  
  In the above screenshot a project named xtm-blt5c972982c0bb40f2-Template has been created automatically, where xtm is the name of the content type, blt5c972982c0bb40f2 is the entry UID, and Template is the name of the XTM template.
5. Now click on the Workflow option from the left navigation panel. You will see the stages of the workflow that you selected while creating the XTM template in Step 1 inside the Workflow management section as shown below:
  
  As we had selected two languages (Chinese and French) while setting up our template in step 1, we are seeing both target languages. You can select the target language from the Target language dropdown menu from here as well. For our example, we have selected French.
6. Select the username in Translate 1 and the reviewer name in Review 1 for translation as shown below:
7. Now click on Start at the bottom left. The translation process will begin. You need to log out of your project management account and log in with your XTM Linguist account credentials that you must have received while creating an account with XTM.
8. Once you have logged in, you will be inside the Tasks tab by default. Click on the Active tab if you are not already in there. You will see two files: one for the translate 1 and the other one for the review as shown below:
9. Click on the first file and you will see the content of your entry in the Source column. And in the Target column, the translator will enter the translated content manually.
10. Once you have added the translated text, you can click on Finish, in the top right corner, to complete the translation process.
  
  Alternatively, if you want to finish the job, you can click on the file in the Active tab and select Finish tasks. Then, select either to finish this particular task or all tasks in the project.
  
  The moment the task is finished, the second lambda function will be invoked which will then take the translated content from your XTM account and upload it in that particular locale in your entry page.
11. Now go to your entry in Contentstack and select the target locale from the dropdown (French [France] in our example). You will see the translated content and the workflow stage change to Review.