Kalan's Blog

Kalan 頭像照片,在淡水拍攝,淺藍背景

四零二曜日電子報上線啦!訂閱訂起來

Software Engineer / Taiwanese / Life in Fukuoka
This blog supports RSS feed (all content), you can click RSS icon or setup through third-party service. If there are special styles such as code syntax in the technical article, it is still recommended to browse to the original website for the best experience.

Current Theme light

我會把一些不成文的筆記或是最近的生活雜感放在短筆記,如果有興趣的話可以來看看唷!

Please notice that currenly most of posts are translated by AI automatically and might contain lots of confusion. I'll gradually translate the post ASAP

Implementing Air Quality Monitoring Applications with Arduino and ESP32 (1) - Introduction to Sensors

Introduction

Lately, I've been really into various IoT applications. I've bought many Arduino boards and sensors. I have Arduino Uno, Arduino Mega2560, and Arduino Nano x 3. I thought it would be fun to play around with them and see if I can create some interesting applications. It's also a good opportunity to review my high school knowledge of electronics and hardware operations.

Coincidentally, my company organized an internal hackathon with no specific theme, so I decided to work on an application I had been wanting to build for a while - an "Air Quality Monitor". Although it's called an air quality monitor, it only measures the concentration of carbon dioxide (CO2) and temperature/humidity. If additional sensors are available, it wouldn't be a big problem to add them.

The reason behind this idea is that the concentration of carbon dioxide can really affect people's thinking and productivity. I believe many of you have experienced feeling dizzy and having confused thoughts during meetings. There is a high possibility that it is due to high levels of carbon dioxide. If there is poor ventilation and the space is enclosed, the carbon dioxide concentration can rise rapidly, reaching over 2000 ppm within a few minutes.

In addition to the implementation itself, I don't like the feeling of just using pre-built libraries and not knowing what's happening behind the scenes. So, I tried to delve into more details to have a better understanding. (Although I still used libraries for implementation, at least I felt more confident about it 😄)

This series of articles will cover the following topics:

  1. Introduction to Sensors - DHT11 and MH-Z14A
  2. Data Communication - UART
  3. Arduino Pitfalls: Explaining the problems encountered during implementation. Many times, it's not a pitfall, but lack of familiarity with Arduino and hardware.
  4. WiFi: To save debugging time, I purchased an ESP32 development board, which already has built-in WiFi and Bluetooth capabilities.
  5. MQTT: To send data to other devices, I used the lightweight MQTT communication protocol.
  6. Grafana / Web: Since the data is stored in a database, I wanted to display it in a fancy way. Here, I used Grafana + Prometheus and Svelte to visualize the data.

Architecture

For easier explanation, the architecture looks something like this:

Infrastructure

  • Arduino Uno sends commands to the CO2 sensor, and the CO2 sensor sends data back to Arduino through UART communication.

  • Arduino Uno sends CO2 data to ESP32 (also using UART).

  • The DHT11 sensor provides temperature and humidity data, which is then sent to ESP32.

  • ESP32 sends temperature, humidity, and CO2 data to the MQTT Broker using WiFi.

  • There are two applications that subscribe to the events: the analysis server and the app server.

    • The analysis server is responsible for sending data to Prometheus, which is then displayed by Grafana.
    • The app server receives the data, stores it in the database, and provides an API for external use.
  • If the carbon dioxide level exceeds a certain threshold, a Slack notification is sent.

That's the overall structure. Now, let's address some possible questions:

1. Why use Arduino Uno separately? Can't ESP32 handle everything?

The initial idea was to use ESP32 as an intermediary to make the overall architecture cleaner. However, after completing the implementation, it seemed unnecessary. The only thing to note is that some Arduino libraries cannot be used with ESP32, such as SoftwareSerial (a library that allows you to use digital pins as TX and RX pins, which will be discussed further in subsequent articles). If this library is not available, we can only use the built-in HardwareSerial, which allows for only one UART communication.

Another reason is that it looks cooler this way. 😄

2. Why connect the DHT11 to ESP32 instead of Arduino Uno?

As mentioned earlier, if we use Arduino Uno, we need to use UART to send the data to ESP32. To simplify things, I connected the DHT11 directly to ESP32, and luckily, the library itself supports ESP32. Of course, it is also possible to use Arduino Uno for this purpose.

3. Why choose MQTT?

MQTT is a lightweight and compact communication protocol. It has less overhead compared to other protocols, making it suitable for IoT applications where the CPU is not fast and memory is limited. However, its availability may not be as good as other protocols.

4. Why have two databases?

Sharp-eyed readers may have noticed that there are two separate databases for analysis and the app. The main reason is that I am not very familiar with Prometheus. If I had to manually query the data, it would take more time. Another reason is that Postgres SQL syntax has better integration with Node.js, making it more convenient to write. In the future, I may explore Prometheus queries!

Results

The most famous air quality monitor product on the market seems to be AWAIR. In addition to its beautiful UI and the ability to measure temperature, humidity, and carbon dioxide, it can also detect chemicals and PM2.5. However, it is quite expensive, priced at 149USD,whichisequivalentto149 USD, which is equivalent to 4,470 TWD. In comparison, the total cost for my implementation is:

  • DHT11 Temperature and Humidity Sensor: Approximately 60 TWD
  • MH-Z14A CO2 Sensor: Approximately 800 TWD
  • ESP32 Development Board: Approximately 200 TWD
  • Arduino Uno: Approximately 800 TWD (original, using Nano would be even cheaper)

Total: 1,860 TWD

Of course, the appearance is quite simple, but with a little modification to the code, it can be integrated into many applications. For example, it can be used with Google Home or to create a simple mobile app, and so on.

Circuit

Here's a simplified version that can fit into a box:

Circuit

Grafana

Grafana

Web Interface

The UI is quite simple, this is just a demonstration.

Gauge

Introduction to Sensors: DHT11 and MH-Z14A

MH-Z14A CO2 Sensor

To obtain the concentration of carbon dioxide, we first need a sensor. After searching online, I found that to measure the concentration of carbon dioxide, we can use a method called NDIR (Non-Dispersive Infrared) sensing.

The principle is based on the fact that gases absorb specific wavelengths of infrared light. By measuring the absorption of infrared light at a wavelength of 4.26μm, we can calculate the concentration of the gas. The formula for calculation is not our concern for now, so let's skip it. However, it's worth mentioning how this sensor is used.

I found limited Chinese resources online, so I'll provide a brief explanation here. You can also refer to the Datasheet, which is the most detailed one I've found so far.

From this datasheet, we can find a few things:

  1. The MH-Z14A communicates through UART and responds to commands. In addition to measuring carbon dioxide concentration, it also supports zero calibration and range adjustment.

    CommandFunctionNote
    0x86Gas concentration
    0x87Calibrate zero point valueThis sensor has three zero calibration methods
    0x99Calibrate span point value
    0x79Start/stop auto-calibration function of zero point valueAuto-calibration is enabled by default when shipped
  2. The MH-Z14A supports three types of output (quite convenient 😄): UART, PWM, and Analog. You can choose the output method you prefer to obtain the carbon dioxide concentration. For this implementation, I chose UART because it is convenient and does not require additional digital-to-analog conversion.

DHT11 Sensor

The DHT11 is a temperature and humidity sensor. It is low-power and has a simple structure, with only 3 pins. By connecting the Vcc and GND pins, it can start working. What's special about the DHT11 is its data transmission method because it only has one pin as a data bus. How does it transmit temperature and humidity data through just one pin?

The answer is timing. In fact, in hardware, many data communications rely on precise timing control. For example, if I want to retrieve 8 bits of data, I can use 2ms as a unit. By reading every 2ms, I can obtain 8 bits of data after 16ms.

However, to achieve this, we need to consider a few things:

  • When does the sender start transmitting data? In other words, from when should I start counting the 2ms?
  • How should we handle errors during transmission, such as environmental factors, temporary short circuits, or data loss?

These are actually described in the datasheet. Let's take a look together.

Single-bus data format is used for communication and synchronization between MCU and DHT11 sensor. One communication process is about 4ms. Data consists of decimal and integral parts. A complete data transmission is 40 bits, and the sensor sends higher data bits first. Data format: 8-bit integral RH data + 8-bit decimal RH data + 8-bit integral T data + 8-bit decimal T data + 8-bit checksum. If the data transmission is correct, the checksum should be the last 8 bits of "8-bit integral RH data + 8-bit decimal RH data + 8-bit integral T data + 8-bit decimal T data".

According to the datasheet, each communication process takes about 4ms. The data consists of 8 bits of integral relative humidity (RH) data, 8 bits of decimal RH data, 8 bits of integral temperature (T) data, 8 bits of decimal T data, and 8 bits of checksum. The sensor sends the higher bits of data first.

The checksum answers the second question. The checksum is calculated by summing up all the data and taking the last 8 bits. (humidity + humidityDec + temp + tempDec != parity)

By comparing the calculated checksum with the received checksum, we can identify any problems during transmission. Of course, there is still a possibility that transmission problems could occur, resulting in the same checksum. In such cases, we can only accept it (although the probability is relatively low in normal environments).

How does the DHT11 transmit data?

To answer the first question, we can refer to the diagram in the datasheet, which explains the entire communication process.

Screenshot from 2020-07-21 21-34-22

  1. When sensing temperature (when Arduino requests data from DHT11), we first send a HIGH -> LOW signal and maintain it for at least 18ms. Then, we change the signal to LOW -> HIGH. In Arduino, we can achieve this using the delay() and digitalWrite functions. P.S. I'm not sure why the DHT11 needs that long to detect the signal. I would appreciate it if hardware experts could provide some insight.
  2. After 20us, the MCU pulls the signal from HIGH back to LOW. (Remember to set the Arduino pin to INPUT using pinMode(PIN, INPUT).)
  3. At this point, the signal should be LOW, and we wait for it to become HIGH after 80us. Then, after another 80us, we bring the signal back to LOW.
  4. Data transmission begins (continued in the next figure).

Screenshot from 2020-07-21 21-44-40

Finally, data transmission begins! The datasheet explains how to distinguish between 0 and 1. As shown in the figure, if the voltage remains high for about 26-28us, it represents a response of 0. If the voltage remains high for 70us, it represents a response of 1. Between each bit, the voltage is brought low for 50us and then back to high. This is done to clearly separate each bit and minimize the chance of misinterpretation during reading.

To calculate this time, we can use the Arduino micros() function. Here's a simple implementation:

void blockUntil(int state)
{
  while (digitalRead(PIN) != state)
  {
  }
}

void main() {
  auto timeS = micros();
  blockUntil(LOW);
  auto timeE = micros();
  auto result = timeE - timeS > 60 ? 1 : 0;
}

I used 60us instead of the 50us described in the datasheet because there can be some errors if the timing is too precise. Now, I'll share my speculation, and I welcome feedback from experts.

I'm wondering if it's because when we run our code, it is compiled into machine code, which means the CPU also consumes some instruction cycles during execution. This accumulation of cycles could make the 50us inaccurate. However, if that were the case, we should adjust the 50us to be smaller rather than larger. So, another possibility I'm considering is that the DHT11 itself is slow.

After completing the entire process, we should have the data! Although I used a library for this implementation, I plan to write another article on how to achieve the same functionality without using a library. If you are willing to spend time reading the datasheet, the implementation should not be too difficult.

Summary

Today, we introduced the sensors needed for this application, their purposes, and provided an overview of the datasheets. Now, you should have a better understanding of how these two sensors transmit data. In the next article, we will explain the common communication channel between hardware devices - UART, share the problems encountered during implementation, and how they were resolved (spoiler: it involved using libraries!).

Prev

The Journey to Data Science from Scratch

Next

Creating an Air Quality Monitoring Application with Arduino and ESP32 (2) - Data Communication UART

If you found this article helpful, please consider buy me a drink ☕️ It'll make my ordinary day shine✨

Buy me a coffee