Because applications and components in the global IoT ecosystem are increasingly interrelated, it becomes essential to be able to connect them to achieve automation.
Advanced systems connectivity allows you to eliminate repetitive scheduling tasks and therefore allows free time for research and development.
One of the tools to make different systems interact is Node-RED – a simple programming tool to link online services, hardware devices, and APIs in an interactive and visually appealing way. It provides a browser-based editor and a broad set of nodes to create and deploy multiple functional streams.
Within this article, we’ll look at a solution we’ve implemented for fast delivery of Node-RED and we’ll deploy a sample stream from the Twitter streaming data analyzer to store and alert about posts from any topic you’re interested in.
A simple way to run the Node-RED instance inside a cloud
On MilesWeb Cloud, the ready-to-use Node-RED development environment is provided as a one-click installation package.
All the features and settings required to run Node-RED on MilesWeb Cloud are declared inside the manifest.jps package file, where the main points to consider are:
- nodered/node-red-docker Image Docker with the most recent tag is used as the solution base
- The created Node-RED instance is assigned 16 dynamic cloudlets, which allow vertically scaling up to 2 GiB of RAM and 6.4 GHz of CPU.
- The automatically attached endpoint allows access to the Node-RED administration panel through a randomly allocated port in Shared Load Balancer (eliminating the need to purchase the public IP address).
So let’s bring it to life – login to your MilesWeb Cloud account and continue with the following steps.
1. On the main panel page, click Import at the top and change to the URL tab. Here, insert the link to the corresponding manifest.jps file:
Https://github.com/jelastic-jps/node-red/blob/master/manifest.jps
Click Import to continue.
2. In the Open Installation dialog box:
- Enter the name of the Environment to be used as your internal hostname
- Optionally, specify the Display Name of the environment (called alias, for display on the panel and on the SSH terminal)
- Select a Region environment (if there are multiple available)
Click Install and wait a moment to display a notification message about completing the process.
Click Open in the browser to access the Node-RED development environment and start creating your first project (the so-called Flow).
In addition, we’ll learn how to interact with this wizard through a practical example of the Twitter content crawler – creating such a useful and handy tool will also help you get started with Node-RED as fast as possible.
Deploying the Twitter Feed Analyzer as Node-RED Flow
So, the application we are going to build is intended to search through the Twitter feed on a specified criteria (keyword, hashtag and/or username), perform sentiment analyzes of observed postings, and process them accordingly. Depending on the character of the message, the action taken should be:
- Detecting negative content – alerting by email message
- For positive and neutral results – store a mention within a dedicated file within a container, for each feature type separately
For analysis, we will use hashtags #Cloud, #CloudComputing, and #PaaS as a basis for investigation of the corresponding hosting area. Obviously, you are free to specify any other search parameters as per your requirements – for example, the name of your company.
The entire structure of the application will look like this:
Here, the following types of components (nodes) are involved:
- Twitter – Used to authenticate with your Twitter account and specify search keywords
- Feeling – Designed to analyze posts and return a corresponding value (negative, positive or neutral)
- Switch – To evaluate the feeling scores and to perform a corresponding action according to the defined tone
- Two Node Files – Provided for storing both positive and negative messages
- Email – Allows you to receive instant alert messages with negative feedback
In total, this flow allows you to react quickly to negative feedback while simultaneously learning about user experience, best practices, news and events in neutral and positive posts. In this way, building such a tool will help you get relevant information on time and be aware of what is happening in the market.
Then, below, we will consider the parameters that should be adjusted for each of these types of nodes and, consequently, we will implement the workflow described above.
Twitter Node
Inside the Node-RED flow editor, expand the social section and drag and drop the twitter icon to place it on the sheet. This will be a good part of the basis of our application.
Double-click the chosen node and specify the following information in the open dialog box:
- Twitter ID – Click the pencil icon to Authorize the application with the login and password of your Twitter account; after this operation has been successful, select Add to end
- Search – Choose an area to conduct searches – for example, all public tweets
- To (for) – Specify a search term (keyword, hashtag and/or username); several entries can be separated with spaces (for the bot to look for the joint mention) or commas without spaces (to detect separate terms)
- Name – Optionally give the node a custom name (or leave the default)
Select Done just above to proceed.
Tip: For a more comprehensive feedback analysis, attach an additional debugging node to the twitter main, so that the posts you get can be reviewed directly by the publisher.
As a result, records of all semantic types will be available within the sidebar debugging tab and are stored within your default msg.payload record.
Sentiment Node
Now, expand the analysis section and choose a sentiment node – it will analyze the found posts and return a corresponding sentiment value to determine if a tweet is negative, positive, or neutral.
This component does not require additional adjustments unless you want to change its name (otherwise, double-click this block, specify the preferred denomination, and click Done).
Switch Node
Now let’s add another switch element from the function section. It is intended to evaluate the feeling score received and to take a corresponding action according to the defined tone.
Tip: The average sentiment index ranges from -5 (which means a very negative tweet) to +5 (a very positive feedback), 0 being a neutral value. Scores outside this range are considered as extreme values.
To adjust the switch node settings, double-click on it and provide the following details in the open dialog box:
- Name – type the name of the custom node (or leave the default)
- Property – Specify the value sentiment.score and + add the following rules to determine which sentimental value should be considered neutral, positive or negative respectively:
- == 0
- > 0
- <0
Select Check All Rules in the bottom box and click Done.
File Nodes
Now let’s set up the register of the neutral and complementary messages observed. But before entering the corresponding node, you need to create the file (s) for such mentions to be stored – then open the Configuration Manager for your Node-RED container and enter the desired destination directory.
Here, select New file from the options menu, enter a name for it (for example, neutral) and press Enter. Repeat these actions to create another file for positive tweets.
Note: Your application must have sufficient permissions to access and write to the created files. To ensure this, connect to your Node-RED container via SSH Gate as follows:
- Generate a pair of SSH keys
- Add your SSH public key to the control panel
- Access your account via SSH protocol
Once the connection is established, execute the command chown node-red {filename} in the destination directory.
Then expand the section of the storage wizard and drag the composite file node to the worksheet.
Double click on this block to adjust the appropriate parameters:
- Filename – Specify a full path to a destination storage file
- Action – Select attach to file to add new tweets to the end of a file. Optionally, check the following additional options available for this configuration:
- Add a new line (\ n) to each payload? – For a new line to be added to each message (can be switched off, if necessary)
- Create the directory if it does not exist? -To create a new directory if it does not exist
- Name – Enter the index number of the switch node condition needed for tweets with the corresponding semantic value to be written to this file (for example, 1 for neutral postings in this example)
Click Done to confirm. Repeat these operations to configure one more file node that will process and save positive tweets.
Email Node
The last required e-mail component may be located in the social section.
Place it inside the workspace and double-click to provide the following details:
- To – enter an email address, alerts on negative tweets should be sent to him
- Server and Port – leave the default values (host gmail.com and corresponding port number 465)
- User Id & Password – Provide credentials to an email account from which notifications will be sent
- Name – Type the index number of the condition of the corresponding node switch according to the type of publication you would like to receive the alerts (in our case, the semantic range for negative tweets was set to 3)
Note: To ensure that the email node will definitely work correctly, your email account (that is, the one from which the alerts will be sent) must meet the following requirements:
- Have the least secure Application Access setting turned on
- Be protected with a strong 8-character password (containing uppercase/lowercase letters, numbers, and symbols)
When finished, click Done.
Note: Because of the inherent limits of the Twitter API, you can view the warning of the rate limit, displayed under the twitter node, if you change node settings and/or click Deploy very often. This does not affect the actual flow operability – just wait a minute for this warning to be replaced with your search word (s) instead.
Checking Results
Then your handmade Twitter bot started working, trawling the current posts and processing the information you asked about. Subsequently, the results of this analysis can be verified as follows:
- For neutral/positive mentions – click Configuration for the Node-RED instance in your MilesWeb Cloud panel and navigate to the appropriate file; its content can be reviewed only through the built-in file manager:
- For Negative Tweets – Just scan new entries inside your email box
That’s it! The Node-RED stream is now ready and working to analyze Twitter broadcast data.
Do you have questions about the Node-RED development environment? Feel free to ask in the comments below.
