BirdNET to MQTT publishing for Home Assistant consumption

birdnet_to_mqtt.py

#! /usr/bin/env python3
# birdnet_to_mqtt.py
# fork of https://gist.github.com/deepcoder/c309087c456fc733435b47d83f4113ff
#
# 20240425
#
# monitor the records in the syslog file for info from the birdnet system on birds that it detects
# publish this data to mqtt
#

import time
import re
import random
import json
import paho.mqtt.client as mqtt
import flickrapi

# Configuration
mqtt_username = 'YOUR_MQTT_USERNAME'
mqtt_password = 'YOUR_MQTT_PASSWORD'
mqtt_server = "YOUR_MQTT_SERVER_IP"
mqtt_topic_confident_birds = 'YOUR_MQTT_TOPIC'
bird_lookup_url_base = 'http://en.wikipedia.org/wiki/'

# Initialization
last_message_time = 0
last_image_url = None
last_sciname = ""

# Flickr API Configuration
FLICKR_PUBLIC = 'YOUR_FLICKR_PUBLIC_KEY'
FLICKR_SECRET = 'YOUR_FLICKR_SECRET_KEY'
flickr = flickrapi.FlickrAPI(FLICKR_PUBLIC, FLICKR_SECRET, format='parsed-json')

# Function to query the Flickr API with extended debugging
def get_flickr_photo_url_with_sciname_in_title(sciname, per_page=100, sort='relevance'):
    print(f"Searching for images for: {sciname}")  # Debugging: Start search
    extras = 'url_c,url_l,url_o,title'
    rand_page = random.randint(1, 100)
	
    license_ids = "1,2,3,4,5,6,7"  #Select the CC-Licences you need
    photos = flickr.photos.search(text=sciname, license=license_ids, page=rand_page, per_page=per_page, extras=extras, sort=sort)['photos']['photo']
    
    filtered_photos = [photo for photo in photos if sciname.lower() in photo.get('title', '').lower()]
    print(f"Found matching images: {len(filtered_photos)}")  # Debugging: Number of filtered photos
    
    if filtered_photos:
        photo = random.choice(filtered_photos)
        for size in ['url_c', 'url_l', 'url_o']:
            if size in photo:
                print(f"Selected image URL: {photo[size]}")  # Debugging: Selected URL
                return photo[size]
    else:
        print("No matching image found.")  # Debugging: No image found
    return None



# Establish MQTT connection
mqttc = mqtt.Client(mqtt.CallbackAPIVersion.VERSION1,'birdnet_mqtt')
mqttc.username_pw_set(mqtt_username, mqtt_password)
mqttc.connect(mqtt_server, 1883)
mqttc.loop_start()

def file_row_generator(s):
    s.seek(0,2)
    while True:
        line = s.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

syslog = open('/var/log/syslog', 'r')
re_bird_found = re.compile(r'python3.*\.mp3$')

while True:
    for row in file_row_generator(syslog):
        current_time = time.time()
        if re_bird_found.search(row) and (current_time - last_message_time >= 60):
            parts = row.split(';')
            if len(parts) >= 6:
                date_part, time_part, sciname, comname, confidence = parts[0].split()[-1], parts[1], parts[2], parts[3], parts[4]
                ts = date_part + " " + time_part
                
                if sciname != last_sciname:
                    new_image_url = get_flickr_photo_url_with_sciname_in_title(sciname)
                    # Check if a new URL was found
                    if new_image_url:
                        print(f"New image URL found: {new_image_url}")  # Debugging: New URL found
                        last_image_url = new_image_url
                    else:
                        print("No new image URL found, using the old one.")  # Debugging: Using old URL
                    last_sciname = sciname

                bird = {
                    'ts': ts,
                    'sciname': sciname,
                    'comname': comname,
                    'confidence': confidence,
                    'url': bird_lookup_url_base + sciname.replace(' ', '_'),
                    'image_url': last_image_url if last_image_url else "No image available",
                }

                json_bird = json.dumps(bird)
                mqttc.publish(mqtt_topic_confident_birds, json_bird, 1)
                last_message_time = current_time

gistfile1.txt

Special thanks to @deepcoder for the original code!

I've made a few changes, primarily because I just couldn't get the original code to work. It might be because I'm using Nachtzuster's fork of Birdnet (https://github.com/Nachtzuster/BirdNET-Pi).

My changes:

-I've streamlined the code to only include birds that meet a certain threshold, which is all that I need.
-I've minimized the use of Regular Expressions to avoid potential errors, as I find them too error-prone for my liking (or Regex skill, probably).
-To accommodate paho-mqtt version 2.0, I've included the version number in the mqtt.Client command.
-I've added functionality for querying the MQTT username and password.
-The code now retrieves URLs for bird images from Flickr. [It generates a list of approximately 100 bird photos that include the scientific name in the title, and selects one at random. To prevent flooding the MQTT with messages, I've restricted the transmission of information to once per minute. If the same bird is detected multiple times in succession, the image remains the same.]


Make sure to replace YOUR_MQTT_USERNAME, YOUR_MQTT_PASSWORD, YOUR_MQTT_SERVER_IP, YOUR_MQTT_TOPIC, YOUR_FLICKR_PUBLIC_KEY, and YOUR_FLICKR_SECRET_KEY with your actual configuration details.


##Other code
#For the Home Assistant configuration yaml
mqtt:
  sensor:
    - name: "Bird Timestamp"
      state_topic: "birdpi"
      value_template: "{{ value_json.ts }}"
      icon: mdi:calendar-clock
    - name: "Bird Scientific Name"
      state_topic: "birdpi"
      value_template: "{{ value_json.sciname }}"
      icon: mdi:format-letter-case
    - name: "Bird Common Name"
      state_topic: "birdpi"
      value_template: "{{ value_json.comname }}"
      icon: mdi:bird
    - name: "Bird Confidence"
      state_topic: "birdpi"
      value_template: "{{ value_json.confidence }}"
      unit_of_measurement: "%"
      icon: mdi:check-circle-outline
    - name: "Bird Info URL"
      state_topic: "birdpi"
      value_template: "{{ value_json.url }}"
      icon: mdi:earth
    - name: "Bird Image URL"
      state_topic: "birdpi"
      value_template: "{{ value_json.image_url }}"
      icon: mdi:image
    - name: "Bird Information"
      state_topic: "birdpi"
      value_template: >-
        {% set current_time = now().strftime('%H:%M') %}
        {% set confidence = value_json.confidence | float %}
        {{ current_time }}: {{ value_json.comname }} ({{ value_json.sciname }}; {{ (confidence * 100) | round(0) }}%)


#For lovelace:

type: picture-entity
show_state: true
show_name: false
camera_view: live
entity: sensor.bird_information
camera_image: camera.birdimagewikipedia
name: 'Last Bird:'


#For lovelace part 2:

type: custom:popup-card
entity: sensor.bird_information
dismissable: true
card:
  type: iframe
  url: http://raspberry3bplus.local/ #or use ip for FullyKiosk (it has problems with text-adresses)
  aspect_ratio: 75%
size: wide







Original Information:
Here is a slight hack python3 program that monitors the syslog output from the BirdNET-Pi bird sound identifications and publishes these records to MQTT. This can be used in Home Assistant to create sensors for the bird identifications.

The program can publish both the output of all birds that BirdNet thinks it hears and also just those above the confidence level that you have set in BirdNET-Pi.

You will have to edit the attached python program 'birdnet_to_mqtt.py' to change the MQTT server, MQTT topics and whether to publish all or just the birds heard above the confidence level.

NOTE: there are different python 'environments' I am not using the birdnet python environment, so any python libraries that are used here are install as root and are used by the python3 binary at: /usr/bin/python3

You will need a couple non-standard python libraries:
```
sudo pip3 install dateparser
sudo pip3 install paho-mqtt
```
copy the python program to /usr/local/bin

```
sudo cp birdnet_to_mqtt.py /usr/local/bin

```
setup a service to run the python program at startup

```
sudo vi /etc/systemd/system/birdnet_mqtt_py.service
# content for this file :
[Unit]
Description=BirdNET MQTT Publish
[Service]
Restart=always
Type=simple
RestartSec=5
User=root
ExecStart=/usr/bin/python3 /usr/local/bin/birdnet_to_mqtt.py
[Install]
WantedBy=multi-user.target

```

enable the python program as a service

```
sudo systemctl enable birdnet_mqtt_py.service
sudo systemctl start birdnet_mqtt_py.service
sudo systemctl status birdnet_mqtt_py.service
```

MQTT sensors in Home Assistant:

```
# birdpi
  - name: "BirdPiNET Bird Heard 01"
    unique_id: "BirdPiNet-01-heard"
    state_topic: "birdpi/confident"
    value_template: >
      {% if value_json.ts is defined %}
        "{{ value_json.ts }}"
      {% else %}
        "NA"
      {% endif %}
    json_attributes_topic: "birdpi/confident"

  - name: "BirdPiNET Bird All 01"
    unique_id: "BirdPiNet-01-all"
    state_topic: "birdpi/all"
    value_template: >
      {% if value_json.ts is defined %}
        "{{ value_json.ts }}"
      {% else %}
        "NA"
      {% endif %}
    json_attributes_topic: "birdpi/all"

```

example lovelace 

```
# https://github.com/gadgetchnnel/lovelace-home-feed-card
    - type: custom:home-feed-card
      title: BirdPiNET-Confident-01
      show_empty: true
      more_info_on_tap: true
      scrollbars_enabled: false
      max_item_count: 10
      compact_mode: true
      id_filter: ^home_feed_.*
      entities:
      - entity: sensor.birdpinet_bird_heard_01
        more_info_on_tap: true
        include_history: true
        remove_repeats: false
        max_history: 10
        content_template: '{{comname}} {{sciname}} {{confidence}} {{ts}}'
        icon: mdi:bird

```

JSON that is generated:

```
{"ts": "1687047081.0", "sciname": "Baeolophus inornatus", "comname": "Oak Titmouse", "confidence": "0.7201002", "url": "http://en.wikipedia.org/wiki/Baeolophus_inornatus"}

```