❌

Reading view

There are new articles available, click to refresh the page.

(D241127) KVM QEMU libvirt virsh virt-install CLI on Ubuntu Linux

✇Phong Lan Tech

Trying to have virtual machines on Ubuntu Linux, of course I'm more comfotable on FreeBSD Bhyve.

virt-install

provision new virtual machines

# virt-install --name vm0 --memory 1024 --vcpus 2 \
--os-variant ubuntu24.04 \
--disk size=10 --network bridge=br0 --graphics none --console pty,target_type=serial \
--location /usr/local/etc/iso/ubuntu-24.04.1-live-server-amd64.iso,kernel=casper/vmlinuz,initrd=casper/initrd \
--extra-args 'console=ttyS0,115200n8 serial'

virsh

management user interface

virsh list --all

virsh start vm0

virsh autostart vm0

virsh autostart --disable vm0

virsh shutdown vm0

virsh destroy vm0

virsh undefine vm0 --remove-all-storage

usb passthrough

$ lsusb   
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 2c7c:0125 Quectel Wireless Solutions Co., Ltd. EC25 LTE modem
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
$ cat usb_ec25.xml 
<hostdev mode='subsystem' type='usb' managed='yes'>
    <source>
        <vendor id='0x2c7c'/>
        <product id='0x0125'/>
    </source>
</hostdev>
# virsh attach-device vm0 --file usb_ec25.xml --config

Attaching and Updating a Device with virsh

(D241122) godns webhook coturn and dynamic IP address on FreeBSD

✇Phong Lan Tech

This is for FreeBSD. It may not work on Linux because sh and sed on FreeBSD are not the same as Linux (bash and other sed).

We're running a STUN/TURN server, it work well but we don't have a static IP address. Everytime we got a new IP address we need to change coturn config and restart the service.

There is a way to get it work by a cron job that run a shell script check public ip every 5 minutes and restart service if the public IP address has changed.

#!/bin/bash

# Linux only, doesn't work on FreeBSD
current_external_ip_config=$(cat /etc/turnserver.conf | grep "^external-ip" | cut -d'=' -f2)
current_external_ip=$(dig +short <MY_DOMAIN>)

if [[ -n "$current_external_ip" ]] && [[ $current_external_ip_config != $current_external_ip ]]; then
        sed -i "/^external-ip=/ c external-ip=$current_external_ip" /etc/turnserver.conf
        systemctl restart coturn
fi

ref: set up with dynamic ip address

Since we're running a godns daemon to update our IP to Cloudflare DNS server we also want godns send a webhook to coturn server whenever it update IP to Cloudflare. That may be more effective.

So this is what we have:

godns

$ cat /etc/godns/config.json 
{
  "provider": "Cloudflare",
  "login_token": "YOUR_TOKEN",
  "domains": [
    {
      "domain_name": "yourdomain.com",
      "sub_domains": [
        "@"
      ]
    }
  ],
  "ip_urls": [
    "https://api.ipify.org"
  ],
  "ip_type": "IPv4",
  "interval": 300,
  "resolver": "8.8.8.8",
  "webhook": {
    "enabled": true,
    "url": "http://your.coturn.webhook.endpoint:9000/hooks/godns",
    "request_body": "{ \"domain\": \"{{.Domain}}\", \"ip\": \"{{.CurrentIP}}\", \"ip_type\": \"{{.IPType}}\" }"
  }
}

webhook

$ cat /usr/local/etc/webhook.yaml
---
# See https://github.com/adnanh/webhook/wiki for further information on this
# file and its options.  Instead of YAML, you can also define your
# configuration as JSON.  We've picked YAML for these examples because it
# supports comments, whereas JSON does not.
#
# In the default configuration, webhook runs as user nobody.  Depending on
# the actions you want your webhooks to take, you might want to run it as
# user root.  Set the rc.conf(5) variable webhook_user to the desired user,
# and restart webhook.

# An example for a simple webhook you can call from a browser or with
# wget(1) or curl(1):
#   curl -v 'localhost:9000/hooks/samplewebhook?secret=geheim'
- id: godns
  execute-command: "/usr/local/etc/godns.sh"
  command-working-directory: "/usr/local/etc"
  pass-arguments-to-command:
  - source: payload
    name: domain
  - source: payload
    name: ip
  - source: payload
    name: ip_type
  trigger-rule:
    and:
      - match:
          type: value
          value: "your.domain.com"
          parameter:
            source: payload
            name: domain

shell script

$ cat /usr/local/etc/godns.sh
#!/bin/sh

# write ip log to a file
now="$(date +'%y%m%d%H%M%S%N')"
echo $now $1 $2 $3 >> godns.txt

# restart coturn when ip changed
turnserver_config="/usr/local/etc/turnserver.conf"
current_external_ip_config=$(cat $turnserver_config | grep "^external-ip" | cut -d'=' -f2)
current_external_ip_webhook=$2

if [ -n "$current_external_ip_webhook" ] && [ $current_external_ip_config != $current_external_ip_webhook ]; then
        sed -i .old -e "s/external-ip=$current_external_ip_config/external-ip=$current_external_ip_webhook/g" $turnserver_config
        service turnserver restart
fi

It may not work well enough, if something happen and godns can not send webhook to coturn server. But for now we stick with it.

(D241106) OpenWrt virtual machine on FreeBSD bhyve if_bridge tap (vmnet), netgraph, VALE and vether

✇Phong Lan Tech

WRITING

We're buiding our all-in-one server for our business.

The network is complicate when we want to run serveral virtual machine and still want to connect to outside. The router will be virtualized, it run OpenWrt or OPNsense as we want.

We have several choice:

1. Traditional way

if_bridge and tap.

if_bridge and tap

It was not so efficiently.

We test on N4100 CPU:

Client Server Speed
vm2 (Debian) vm0 (ImmortalWrt) 2.11 Gbits/sec
vm2 (Debian) vm1 (Debian) 2.17 Gbits/sec
vm2 (Debian) host (FreeBSD) 1.35 Gbits/sec

Our config:

$ cat /etc/rc.conf
cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="dhcp addm em0 addm tap0"
ifconfig_em0="up"
$ cat /boot/loader.conf
if_bridge_load="YES"
if_tap_load="YES"
$ cat /pool/vm/openwrt/openwrt.conf 
loader="uefi"
cpu=2
memory=512M
network0_type="e1000"
network0_switch="public"
network0_device="tap0"
disk0_type="ahci-hd"
disk0_name="immortalwrt-23.05.4-x86-64-generic-ext4-combined-efi.img"

2. More modern way

vale and vether.

Of course there is epair that we can connect between host and vm.

vale and vether

We test on N4100 CPU:

Client Server Speed
vm2 (Debian) vm0 (ImmortalWrt) 3.50 Gbits/sec
vm2 (Debian) vm1 (Debian) 5.30 Gbits/sec
vm2 (Debian) host (FreeBSD) 1.48 Gbits/sec

Our config:

$ cat /etc/rc.conf
defaultrouter="192.168.1.1"
cloned_interfaces="vether0"
ifconfig_vether0="192.168.1.2/24 up"
$ cat /boot/loader.conf
if_vether_load="YES" 
$ cat /pool/vm/openwrt/openwrt.conf 
loader="uefi"
cpu=2
memory=512M
network0_type="e1000"
network0_switch="public"
disk0_type="ahci-hd"
disk0_name="immortalwrt-23.05.4-x86-64-generic-ext4-combined-efi.img"

Need to run following script everytime after openwrt vm start (connect host to vm). If you have vm-bhyve you can put it to prestart, remember to chmod +x for it.

valectl -h vale-name:vether-name, for example valectl -h vale0:vether0.

3. The most complex thing

netgraph.

netgraph ng_bridge ng_eiface

We test on N4100 CPU:

Client Server Speed
vm2 (Debian) vm0 (ImmortalWrt) 2.41 Gbits/sec
vm2 (Debian) vm1 (Debian) 2.47 Gbits/sec
vm2 (Debian) host (FreeBSD) 2.27 Gbits/sec

ng_bridge is much faster than vether in this case: vm to host.

We think we could keep VALE as our vm switch, and replace vether by netgraph or epair?

netgraph as host virtual interface and vale as bridge, report bad pkt??? something was not right. we will look into it further and do it manually.

Client Server Speed
vm2 (Debian) vm0 (ImmortalWrt) 3.49 Gbits/sec
vm2 (Debian) vm1 (Debian) 5.25 Gbits/sec
vm2 (Debian) host (FreeBSD) 1.46 Gbits/sec

Ref:

$ cat /boot/loader.conf
ng_eiface_load="YES"
ng_bridge_load="YES"
ng_ether_load="YES"
$ cat /etc/rc.conf
ngbridge_enable="YES"
ngbridge_names="lan"
ngbridge_lan_eifaces="nge_1u"
ngbridge_nge_1u_mac="00:37:92:01:02:02"
ngbridge_nge_1u_addr_num="1"
ngbridge_nge_1u_addr_1="inet 192.168.1.2/24"
ngbridge_lan_eifaces_keep="nge_1u"
ngbridge_lan_route_num=1
ngbridge_lan_route_1="-net default 192.168.1.1"
ngbridge_lan_vlans="NO"
$ cat /pool/vm/openwrt/openwrt.conf 
loader="uefi"
cpu=2
memory=512M
network0_type="e1000"
network0_switch="lanbridge"
disk0_type="ahci-hd"
disk0_name="immortalwrt-23.05.4-x86-64-generic-ext4-combined-efi.img"

(D241101) Ghi chép về việc triển khai nhúng live streaming camera video lên website

✇Phong Lan Tech

BÀI ĐANG VIẾT

Đây là những ghi chép trong quá trình triển khai nhúng live streaming camera video lên website trangtraihuounai.com.

1. Yêu cầu

Không quá nhiều điểm đặc biệt:

  • Nhúng livestream camera của trang trại lên website
  • Hệ thống đơn giản, có thể dễ dàng mở rộng khi cần thiết

2. Ý tưởng

Gọn nhẹ nhất có thể:

  • Sử dụng máy tính all-in-one cho tất cả các tác vụ
  • Lấy luồng RTSP từ camera phát lại bằng go2rtc
  • Nhúng trực tiếp vào website hoặc phát kênh Youtube?

2.1. Thiết bị thử nghiệm

Đang có sẵn các thứ sau:

  • Mini PC N4100: 4x1GB Ethernet
  • PoE Switch không biết tên
  • Imou Cruiser 2 (5MP Outdoor Wi-Fi P&T Camera)

Thay thế thiết bị khác sau khi chạy thử:

  • All-in-one box: HP / Dell
  • Camera: Dahua / Hikvision

2.2. Sơ đồ hệ thống

Sơ đồ hệ thống

Sẽ tách router trong tương lai, hiện tại chạy ImmortalWrt trên môi trường ảo hóa.

2.3. Các bước triển khai

Task List

  • [x] Cài đặt FreeBSD box
  • [x] Cấu hình hệ thống mạng sẵn sàng cho máy ảo
  • [x] Cài đặt máy ảo chạy ImmortalWrt làm router
  • [x] Cấu hình ImmortalWrt, quy hoạch VLAN
  • [x] Cài đặt máy ảo chạy Debian làm docker host
  • [x] Cài đặt nginx làm reverse proxy, nat port 80/443 từ router
  • [x] Cài đặt go2rtc lấy luồng RTSP / ONVIF từ camera
  • [x] Cài đặt nginx làm web server, nhúng stream vào trang web
  • [x] Chuyển thiết bị đến trang trại, quay pppoe mở port 80/443
  • [ ] Chạy thử trong 30 ngày

3. Các vấn đề gặp phải

  • high latency from cameras to web media player: check bridge+tap on bhyve, maybe passthru all nics and use something like vether / epair / vale / netgraph.. done.
    • Deploy TURN server and stream by WebRTC, much better latency
  • transcoding video requires high CPU load, checked bhyve GPU passthru but it didn't work. moved to Linux instead of FreeBSD. needed something like CUDA
  • can not pull rtsp stream from EZVIZ camera, they use their own protocol. best camera brand for this job: Dahua, Imou
  • security risk when expose go2rtc API
    • Deploy a mediamtx WebRTC endpoint, expose via a reverse proxy

4. New proposal

4.1. Network

Router on baremetal:

  • HINLINK OPC-H28K Board run KWrt
  • DHCP Static Mappings for services and cameras

4.2. Server

Hardware:

Software:

(D241024) IP PBX call distributor VoIP Gateway GSM/3G/4G/5G for home and small office

✇Phong Lan Tech

PROOF OF CONCEPT

1. Requirement and concept

We're building a PBX system for our business. It will handle:

  • automatic distribute incoming calls
  • call outside by specific phone numbers, automatic route to reduce call cost
  • sms send api for OTP maybe?

Flow Incoming Call

Since we don't have any SIP trunk from our carriers, we will take incoming calls from regular sim cards.

  • Disadvantage:
    • Single-line: only one concurrent call per SIM card at a time
    • Need more work to handle system failure due to OS, power..
  • Not sure:
    • Voice over LTE (VoLTE) are supported by all carriers.
    • Voice over Wi‑Fi (VoWiFi) are supported by VinaPhone, Mobifone?

Call flow as bellow: GSM PBX diagram

Our stack:

2. Step by step

2.1 Asterisk Installation

  • OS: Debian 12 ("Bookworm")
$ mkdir tmp
$ cd tmp/
$ wget https://downloads.asterisk.org/pub/telephony/asterisk/asterisk-22-current.tar.gz
$ tar zxvf asterisk-22-current.tar.gz
$ cd asterisk-22*/
$ sudo ./contrib/scripts/install_prereq install
$ ./configure
$ make menuselect

Most of IP phones are support G722 and / or G729 codecs now.

  • Under Add-ons:
    • Select [*] format_mp3
  • Under Codec Translators:
    • Select [*] codec_opus
    • Select [*] codec_silk
    • Select [*] codec_siren7
    • Select [*] codec_siren14
    • Select [*] codec_g729a
  • Under Core Sound Packages:
    • Deselect [*] CORE-SOUNDS-EN-GSM
    • Select [*] CORE-SOUNDS-EN-WAV
    • Select [*] CORE-SOUNDS-EN-G722
    • Select [*] CORE-SOUNDS-EN-G729
  • Under Extras Sound Packages:
    • Select [*] EXTRA-SOUNDS-EN-WAV
    • Select [*] EXTRA-SOUNDS-EN-G722
    • Select [*] EXTRA-SOUNDS-EN-G729
$ make
$ sudo ./contrib/scripts/get_mp3_source.sh
$ sudo make install
$ sudo make config
$ sudo make samples
$ sudo mkdir /etc/asterisk/samples
$ sudo mv /etc/asterisk/*.*  /etc/asterisk/samples/
$ sudo asterisk -rvvvv

Most important config files:

  • extensions.conf
  • pjsip.conf

2.2. Virtual machine vm1

Purpose: VoIP GSM Gateway to handle voice call from and or to 3G/4G/5G MNOs.

  • OS: Debian 12
  • IP: 192.168.1.11
  • Software: Asterisk
  • Driver: asterisk-chan-quectel

2.3. Virtual machine vm0

Purpose: Local PBX.

  • OS: Debian 12
  • IP: 192.168.1.10
  • Software: MikoPBX

3. Resource

Softwares:

4. Miscellaneous

  • Linux KVM is much more complicated than FreeBSD Bhyve so maybe we sould run FreeBSD host and Linux guest
  • Debian has no Asterisk in their repo, we moved our guest to Ubuntu since we don't want to complie Asterisk ourself
  • Run MikoPBX as a Local PBX, Asterisk as a trunk provider. We found that MikoPBX is better than FreePBX in our case.
  • Call quality seems not good enough, need to check jitter buffer again
  • https://github.com/RoEdAl/asterisk-chan-quectel
test@vm0:~$ cat /etc/asterisk/pjsip.conf 
;===============TRANSPORT

[system-udp]
type=transport
protocol=udp
bind=0.0.0.0

;===============TRUNK

[miko0]
type=aor
contact=sip:192.168.1.11:5060

[miko0]
type=endpoint
context=quectel-incoming
disallow=all
allow=ulaw
aors=miko0...

(D241010) Mail Server for small businesses FreeBSD OpenSMTPD and Dovecot

✇Phong Lan Tech

We've been running mail server for a quite of time. The last software was Stalwart Mail Server, unfortunately it's under heavy development. They move so fast then the maintain job is a pain.

We decided to going back to a more mature solution.

OpenSMTPDrspamdDovecot

1. Should know

Some information that we think you should know when you want to run your own mail server.

1.1. Protocols:

1.2. Formats:

1.3. Software components:

1.4. Resources:

2. Software stack

We're running FreeBSD so we want to run our mail server on it. The solution need to be easy to deploy and maintain.

For small businesses we will not store usernames and passwords in LDAP or SQL databases, we store such information in flat-file databases.

2.1. FreeBSD (operating system)

There is nothing to say. FreeBSD is quite boring, it just works ^^.

We're going to run our mail server in a FreeBSD jail (managed by AppJail). We already have HAProxy as our Load Balancer.

FreeBSD is a good operating system. Please donate to their work.

FreeBSD_Hz

We just need to make sure our FreeBSD server is up to date.

# pkg update
# pkg upgrade

Create vmail user and vmail group. This is the user/group that’s used to access the mails.

# pw useradd vmail -u 5000 -d /home/vmail -s /usr/sbin/nologin -m

Get a free TLS/SSL certificate for your domain from a certificate authority (ZeroSSL, Let's Encrypt..) by acme.sh or Certbot.

2.2. OpenSMTPD (SMTP)

We consider Postfix which is more popular but we found that OpenSMTPD is easier to config so we will choose it as our MTA.

We love the config syntax, it remind us about PF.

opensmtpd

2.2.1. Install OpenSMTPD

# pkg install opensmtpd opensmtpd-extras

2.2.2. Config OpenSMTPD

2.2.2.1. smtpd.conf

Please read:

Please read it carefully and make your own config file. It's very important.

Modify your file /usr/local/etc/mail/smtpd.conf:

2.2.2.1.1. TLS/SSL certificate

declare your certificate as follow, pkiname named mail.example.com:

pki mail.example.com cert "/usr/local/etc/certs/example.com/fullchain.pem"
pki mail.example.com key "/usr/local/etc/certs/example.com/key.pem
2.2.2.1.2. Tables

declare your tables

table aliases file:/usr/local/etc/mail/aliases
table virtuals file:/usr/local/etc/mail/virtuals
table domains file:/usr/local/etc/mail/domains
table credentials file:/usr/local/etc/mail/credentials
table secrets file:/usr/local/etc/mail/secrets
table passwds file:/usr/local/etc/mail/passwds

We will have: aliases, virtuals, domains, credentials, secrets and passwds.

See OpenSMTPD tables below for more information.

2.2.2.1.3. Bind
# STARTTLS port 25
listen on 0.0.0.0 port 25 tls pki mail.example.com
# SMTPS port 465
listen on 0.0.0.0 port 465 smtps pki mail.example.com auth <credentials>
# SUBMISSION port 587
listen on 0.0.0.0 port 587 tls-require pki mail.example.com auth <credentials>

Listen to IPv4 only 0.0.0.0.

For SMTPS (port 465), SUBMISSION...

(D240924) ZFS snapshot check holds and release

✇Phong Lan Tech

The following command will show all snapshots of [pool] (<-replace this with your pool name) that have holds

zfs list -H -o name -t snapshot -r pool | xargs -n1 zfs holds -H

the properties will be listed as property:stuff

with that information we can free the snapshots..

zfs list -H -o name -t snapshot -r pool | xargs -n1 zfs holds -H | awk '{print $1}' | xargs -n1 zfs release property:stuff

(replace 'property:stuff' with whatever is holding your dataset)

..and finally delete them

zfs destroy -r [pool]/[dataset][@snapshot]

source: How to check that all ZFS snapshots within a pool are without holds before destroying that pool

❌