Author: David Vassallo

Developing Alexa Skills

~ David Vassallo

In this article we’ll explore how to add some custom skills to your Alexa powered device. The material in this blog post was tested on an Echo Dot – however Amazon recently enabled Alexa on most Android powered smartphones so the barrier for entry to developing these skills has been lowered significantly.

I actually ended up implementing the following features into my custom Alexa skill:

  • Switching on my PC
  • Controlling the TV power and TV source
  • Getting the current Euro value of ethereum

In this article we’ll focus on the second item – controlling the TV, however if you’re interested in exploring any of the other two topics please leave a vote in the poll at the end of the article.

Here’s a video of the results from what we’ll be building here:

The equipment used in this project – apart from Alexa – were the following:

  • A raspberry pi connected via HDMI to the TV. In our case, the Raspberry Pi is loaded with OSMC / Kodi. The raspberry pi also needs a connection to the internet.
  • The TV itself is a Samsung Smart TV. I don’t think this point is very important as i’ll discuss later – we’re including it here for completeness.

Step 1 : Getting to know HDMI-CEC

You have several options when it comes to controlling your TV from the raspberry pi. you could hook into your TV’s API if it has one, or go old-school and use actual infrared to control your TV just like your remote control does – using some very cheap hardware and LIRC on the raspberry PI. However these options are either vendor-dependent (or even worse – firmware version dependent) or are a bit clunky to setup and require line-of-sight. HDMI-CEC is a perfect solution to this. There’s plenty of literature on HDMI-CEC out there on the internet, so we won’t go into the details here – but in a nutshell HDMI cables are not one-way like VGA back in the day, it’s a two way cable that allows the video player to send messages to the TV and vice-versa. Although HDMI-CEC goes by different names depending on the vendor, it is relatively vendor agnostic  and works on multiple TV models.

The installation on the raspberry pi is a simple one liner:

sudo apt-get install cec-utils

The main program you’d use is “cec-client” and it’s best documented in this tutorial blog post:

https://blog.gordonturner.com/2016/12/14/using-cec-client-on-a-raspberry-pi/

I managed to find all the codes I needed from the excellent cec-o-matic site:

http://www.cec-o-matic.com/

This results in the following four commands that we’ll use in our program:

Switch the tv on:
                 'echo "on 0" | cec-client RPI -s -d 1
Switch the tv off:
                 'echo "standby 0" | cec-client RPI -s -d 1'
Change the tv source to kodi:
                  'echo "as" | cec-client RPI -s -d 1'
Change the tv source to satellite:
                  'echo "tx 1F:82:30:00" | cec-client RPI -s -d 1'

We can do a simple echo “as” when switching to kodi because the raspberry pi simply sets itself as Active Source, while when switch to satellite we used cec-o-matic with the following settings:

Capture

Notes:

  • The source is set to Recording 1 (which we got from using cec-client to scan the HDMI bus, like so: 
    echo "scan" | cec-client RPI -s -d 1
  • The destination is set to broadcast
  • The physical address is set to 3.0.0.0, which we also get from the cec-client scan; just look out for the physical address of the HDMI source you would like to put on

Those are the only commands we’ll use in this blog post, but uou can also monitor CEC commands using cec-client to implement pretty much anything:

https://ubuntu-mate.community/t/controlling-raspberry-pi-with-tv-remote-using-hdmi-cec/4250

Step 2: Preparing the Raspberry Pi

Now that we have the cec-client sorted out, we need to have some framework which allows alexa to execute these commands. I’ve seen some implementations use firebase to allow the Pi to communicate with Alexa:

https://medium.com/@vishmathur5/alexa-turn-on-my-tv-bcaccc94f1c2

It’s a pretty neat solution, but a more elegant solution (cause we dont need firebase) is to use the nifty python library “Flask-Ask” from John Wheeler. Amazon have a simple tutorial you can follow step-by-step to get up and running, here:

https://developer.amazon.com/blogs/post/Tx14R0IYYGH3SKT/Flask-Ask-A-New-Python-Framework-for-Rapid-Alexa-Skills-Kit-Development

Once you have setup your memory game, we now proceed to modify our code to include the following functions:

The code should be pretty straightforward: we define two intents, one to switch the TV on/off, and the other to change the source. Each intent accepts one input variable, to help us control the output of the function. Depending on this input variable, we issue a cec-client command which we explored in the previous section, that’s executed directly by the OS using the subprocess module in lines 6, 9, 18 and 21.

NB note that we use hardcoded commands – otherwise having “shell=True” would make these subprocess calls insecure

Once the command is run, we return a statement containing a message which Alexa will read out, which brings us to our next section…

Step 3: Building your Alexa Skill

The final step is to interface your Alexa to the program you just wrote above. Broadly speaking, the following three steps are followed:

  • The user triggers Alexa into listening mode by using the trigger word “Alexa” (by default)
  • Next, the user can speak several phrases that include the invocation name of your program which lets Alexa know that what’s coming next is meant as commands for your program. A word of caution here, make sure to use a clear but relatively unique invocation name. If you use an easily-misunderstood word like “Jarvis”, Alexa might not realise you mean to invoke your program. On the other hand, if you use a common invocation word like “Home Assistant”, probably some other published skill also has that same invocation – leading Alexa to enable that published skill rather than using your program. In my case, I used the invocation word “batman“, which is easily understood by Alexa, and doesn’t have any competing published skills. So in this case the invocation phrases I could use are:
    • Alexa, ask batman to …
    • Alexa, open batman and …
  • Next comes the intent. These map directly to the intents we wrote in our program. In our case, we have two intents, (lines 1 and 13) which are the TVPowerIntent and the TVSourceIntent. Make a note of these names, because we need to instruct the Alexa SDK which utterances or phrases the user can speak to invoke or trigger these intents. For example, we might want the following:
    • Alexa, ask batman to switch on the television => TVPowerIntent, with variable “power” = on

As a summary, we’d end up with the following command structure:

Alexa_CMD_Structure

This mapping is done via the Amazon Developer Console. Make sure to login to the developer console using the same account in which the echo is registered.

If you followed the tutorial for flask-ask, you probably already have a skill defined. Just make sure that the invocation of the skill matches the invocation phrase that you’d like to use. In my case, the skill invocation is “batman”. The following steps assume you are using the Interaction Model Builder.

  • On the left hand side, under intents, click “add new”. We define two intents, with the same names as the intents we defined in flask-ask above

intents

  • For each intent, we define a variable (or slot) to pass into our function. The name of the slot has to match the variables you defined in the @ask.intent lines of flask-ask:

slots

  • Each slot is given a set of accepted values. You can use the built-in alexa values for word, names, or numbers. In my case I defined a custom value set (called “power_states” above, which has only two accepted values, on/off:

slot_values

  • Last, for each intent we define a set of utterances which will trigger our intent – our “intent phrase”. Put in as many variations as you can think of, to make it easier to call your program:

intent_phrases

Note how the name of the variable or slot is {enclosed in curly brackets}.

The steps to make amazon invoke your program written in flask-ask are exactly the same as in the tutorial we referenced previously.

That’s it… save and build your model. (complete generated model code included at the end of the article)

Advertisements

Analyzing credit card transactions using machine learning techniques – 3

~ David Vassallo

Introduction

In a previous article, we explored how PCA can be used to plot credit card transactions into a 2D space, and we proceeded to visually analyse the results. In this article, we take this process one step further and use hierarchical clustering to automate parts of our analysis, making it even easier for our hypothetical financial analyst to find anomalies within their data set (for a review of the data set being used, make sure to check out the first article in our series).

Clustering

 Clustering is an unsupervised machine learning method where data points are grouped together according to a given “distance metric”. Usually this defaults to euclidean distance, though other distance functions exist (such as cosine similarity) depending on your application. There are a large number of clustering algorithms, such as DBSCAN, Hierarchical Clustering, and even a hybrid of the two – HDBSCAN. the Orange library implements hierarchical clustering, so we will use this for demonstration purposes in the rest of this article.

Results

After running the algorithm, and selecting the top 5 clusters, we see the following results:

HC_1

The data points are organised hierarchically into a tree with each tree leaf  being a cluster. The Orange library helpfully splits each cluster into separate colors (on the right hand side), so one can easily discern which data points belong to a cluster.

We will plot these cluster colors onto our original PCA scatter plot in order to better compare the two and more easily absorb the extra information that the clustering algorithm gives us:

HC_3

As expected, the clustering algorithm grouped most of the points in the lower left corner together in bright yellow. We discussed in the previous article about how the PCA algorithm is heavily influenced by the “total” data attribute since it introduced the most variance in our data set, so we expect that the clustering algorithm will also be highly dependent on the “total” attribute.

In the previous article, once we processed our data set using PCA, we noticed one extreme outlier, due to the “total” value of this outlier being far higher than that of the other data points. This is reflected in the clustering results by a cluster containing a single point:

HC_2

The first point is the only point in the light blue cluster, while the entries below it make up another small light red cluster. These entries represent those entries in our data set where the “total” value of the credit card transfer was higher than is usual for other transactions in the data set.

Looking at the next (light green) cluster, we spot an anomaly that managed to make it past our manual analysis in our previous article:

HC_4

The first data point in the cluster is labelled “Harold Benjamin Solicitors“, while all the rest are from the expected “HMCOURTS“. If we check the raw data set, we see that this entry has a total of “2124”, which is well over the average total normal seen in our transactions. Therefore we can now add quite a specific anomaly to our already discovered ones: that an abnormally high payment has been made to “Harold Benjamin Solicitors”, because usually such high payments go to “HMCOURTS”. However, there is a high probability that this observation is a false positive (i.e. not really an anomaly), because we know that solicitors do in fact tend to charge large sums of money, and because this data point is the only transaction in our data set which is going to the solicitors, we have no frame of reference to decide if this is an abnormally high payment in the context of solicitors or not (incidentally this highlights why it is difficult to work with “imbalanced data sets” – or data sets where different classes of data do not have adequate representation.

Conclusion

We’re at the conclusion of our three-part series, and hopefully you now have a better understanding of how correspondence analysis, PCA, and clustering work.

We’ve seen that since correspondence analysis works directly on tabular categorical data, it is easy to apply it to data which is already in the form of a table, and it highlights relationships between the categories in your data. PCA on the other hand, is more heavily influenced by those attributes which introduce the highest variation in your data. It is very good in helping to highlight those anomalies that are present within this highly variant attribute, but you may miss some of the relationships within the data – so it’s always better to investigate your data with more than one algorithm and merge the results in your reporting.

While clustering was also highly dependent on “total” – so it did not give us the more subtle insights which Correspondence Analysis did – it was a very useful addition to plain PCA since it helps an analyst to quickly data into more manageable chunks which makes anomaly detection easier, as was the case with our solicitors payment in the above example.

 

Previous Articles in the Series

Analyzing credit card transactions using machine learning techniques – 1
Analyzing credit card transactions using machine learning techniques – 2