By now all electricity consumers in Norway are supposed to have got their electricity meters replaced with new smart meters. (The smart meters are often referred to as AMS (Advanced Metering System)).The new smart meters register electricity consumption at least every hour and automatically sends it to the power company. There's no more use for manually reporting the usage.What's interesting is that the new smart meters all come with a so-called HAN port (short for Home Area Network). Using that port it's possible to get full access to your own electricity usage realtime. While I'm sure great services (and APIs) for using this data will be provided by both my energy company and third party vendors I didn't want to sit back and wait.(You can today get access to a bit of delayed hourly usage if you log in to https://plugin.elhub.no/. They also have some nice Ajax calls which are easy to understand and tweak.)Step 1 - opening the HAN portBy default the physical HAN ports of the smart meters are closed off and not sending any data. All you need to do is to contact customer support at your power company and they'll quickly open it remotely.My power company - Norgesnett - used almost a month to open it up as they said the newly installed meter had to first be registered in some computer system. Of course I also noticed a pretty bad security vulnerability while at it.👉 Dinside: Sjekk om du kan aktivere HAN-portenStep 2 - getting the hardwareThe smart meters use the M-Bus standard for the physical data transfer. So to read the data stream you need to get some kind of M-Bus converter. The smart meters act as a so-called master and the receiver must be a slave. The master gives enough power to run a slave.Hardware - first tryFrom what I read from the forums a lot of people are using - successfully - this (or similar) M-Bus to USB master/slave found on AliExpress, but for me and others it didn't work. I received shorter packages than expected and only the first part of it was readable for me.Not being an electrical engineer and not knowing how to debug or resolve this I just threw money at the problem and bought another converter.Hardware - second tryAnother commonly used M-Bus to USB master/slave from eBay did the trick for me. I connected it to my good old Raspberry Pi (Model B Rev 2). The HAN port in the smart meters has a RJ-45 connector with the signal being transmitted on pin 1 + 2.So I just used an old network cable to connect the smart meter and converter.Step 3 - reading raw dataPython is not my mother tongue, but it's a language I really like and enjoy writing. It's almost always available on whatever system you're on, and the standard library is pretty extensive.Do a simple pip install pyserial and you're ready to read data from the USB port.The serial port settings for the data stream for me was 2400 baud, parity bit none and byte size 8 bits. So doing something similar to this I got the raw data stream (code works in both Python version 2.7 and 3.4):It would output something similar to this (I have anonymized the data a bit):Step 4 - understanding OBISSo what are those bytes coming from the HAN port? They are following the DLMS (Device Language Message Specification) protocol and are sent inside HDLC frames and contains OBIS (Object Identification System) codes that describes the electricity usage. Everything is part of IEC 62056 which is a set of standards for electricity metering data exchange.How often the messages arrives varies from one meter vendor to another. The same goes for the actual format of the messages. I don't know if there are any other vendors, but at least Aidon, Kaifa and Kamstrup have made smart meters for the Norwegian market, and they all provide documentation for their own OBIS messages.On my Kamstrup meter I get the current power used every 10 second + the total kWh usage every hour.To really understand the HDLC and OBIS codes you need dig into different sources around the Internet, but the Norwegian forums at hjemmeautomasjon.no is a great source of information. There are so many knowledgeable people sharing their work and helping each other out.Realtime usage - every 10 secondI assume I haven't got everything figured out and there are likely some errors, but this is my interpretation of the message:For me the most interesting part of this message is the OBIS for Active Power + (1.1.1.7.0.255) which tells how much power - in Watt - that is currently being used. If you have a house that produces electricity and exports it to the grid (e.g. if you have solar cells) the exported power would appear as the OBIS for Active Power - (1.1.2.7.0.255).Total usage - every hourThe message appearing hourly is similar to the one that comes every 10 second, but contains a bit more information:The only part that I really care about is the Active energy A+ (OBIS code 1.1.1.8.0.255), which is total power usage - in kilowatt hour (kWh) - since the installation of the smart meter. Keeping track of this value one knows the hourly power consumption. This is the value you have to pay for. If you produce and exports power it would appear as Active energy A- (1.1.2.8.0.255).Error detectionError detection is supported through cyclic redundancy check (CRC) in both the header and footer of the frame. In the start there is a header check sequence (HCS), and in the end there is a frame check sequence (FCS). The checksum algorithm used is the CRC-16/X-25. There are libraries for all kinds of programming languages implementing all sorts of checksum calculations. I have used the Python library crccheck which provides the class CrcX25 which takes care of this.Links to more informationHere are some of the sources of information I've used for deciphering the messages. I'll leave them here for anyone wanting to dive deeper:https://www.dlms.com/files/Green-Book-Ed-83-Excerpt.pdfhttps://www.dlms.com/files/Blue-Book-Ed-122-Excerpt.pdfhttps://github.com/roarfred/AmsToMqttBridge/blob/master/Samples/Kaifa/readme.mdhttps://github.com/roarfred/AmsToMqttBridge/blob/master/Samples/Kamstrup/obisdata.mdhttp://www.interfacebus.com/HDLC_Protocol_Description.htmlhttps://www.hjemmeautomasjon.no/forums/topic/390-ny-strømmåler-med-han-interface/?page=4https://www.nek.no/wp-content/uploads/2018/10/Kamstrup-HAN-NVE-interface-description_rev_3_1.pdfhttps://byggebolig.no/imageoriginals/88b3d1774ecb41e6a3fe067ae9e6a893.pdfRounding it upThat's it. In this post I've shown how I connected to the HAN port of my smart meter, how I read the data and how to transform the byte arrays into meaningful information.In the next post I'll discuss how I store the data and calculate the price of the electricity usage.
