In this guest post, Ramona Rayner from our partner Sony shows you how to quickly explore different models and AI capabilities, and how you can easily build applications on top of the Raspberry Pi AI Camera.

The recently launched Raspberry Pi AI Camera is an extremely capable piece of hardware, enabling you to build powerful AI applications on your Raspberry Pi. By offloading the AI inference to the IMX500 accelerator chip, more computational resources are available to handle application logic right on the edge! We are very curious to see what you will be creating and we are keen to give you more tools to do so. This post will cover how to quickly explore different models and AI capabilities, and how to easily build applications on top of the Raspberry Pi AI Camera.

If you didn’t have the chance to go through the Getting Started guide, make sure to check that out first to verify that your AI Camera is set up correctly.

Explore pre-trained models

A great way to start exploring the possibilities of the Raspberry Pi AI Camera is to try out some of the pre-trained models that are available in the IMX500 Model Zoo. To simplify the exploration process, consider using a GUI Tool, designed to quickly upload different models and see the real-time inference results on the AI Camera.

In order to start the GUI Tool, make sure to have Node.js installed. (Verify Node.js is installed by running node --version in the terminal.) And build and run the tool by running the following commands in the root of the repository:

make build

./dist/run.sh

The GUI Tool will be accessible on http://127.0.0.1:3001. To see a model in action:

• Add a custom model by clicking the ADD button located at the top right corner of the interface.
• Provide the necessary details to add a custom network and upload the network.rpk file, and the (optional) labels.txt file.
• Select the model and navigate to Camera Preview to see the model in action!

Here are just a few of the models available in the IMX500 Model Zoo:

Network Name	Network Type	Post Processor	Color Format	Preserve Aspect Ratio	Network File	Labels File
mobilenet_v2	packaged	Classification	RGB	True	network.rpk	imagenet_labels.txt
efficientdet_lite0_pp	packaged	Object Detection (EfficientDet Lite0)	RGB	True	network.rpk	coco_labels.txt
deeplabv3plus	packaged	Segmentation	RGB	False	network.rpk	–
posenet	packaged	Pose Estimation	RGB	False	network.rpk	–

Exploring the different models gives you insight into the camera’s capabilities and enables you to identify the model that best suits your requirements. When you think you’ve found it, it’s time to build an application.

Building applications

Plenty of CPU is available to run applications on the Raspberry Pi while model inference is taking place on the IMX500. To demonstrate this we’ll run a Workout Monitoring sample application.

The goal is to count real-time exercise repetitions by detecting and tracking people performing common exercises like pull-ups, push-ups, ab workouts and squats. The app will count repetitions for each person in the frame, making sure multiple people can work out simultaneously and compete while getting automated rep counting.

To run the example, clone the sample apps repository and make sure to download the HigherHRNet model from the Raspberry Pi IMX500 Model Zoo.

Make sure you have OpenCV with Qt available:

sudo apt install python3-opencv

And from the root of the repository run:

python3 -m venv venv --system-site-packages
source venv/bin/activate
cd examples/workout-monitor/
pip install -e .

Switching between exercises is straightforward; simply provide the appropriate --exercise argument as one of pullup, pushup, abworkout or squat.

workout-monitor --model /path/to/imx500_network_higherhrnet_coco.rpk
 --exercise pullup

Note that this application is running:

• Model post-processing to interpret the model’s output tensor into bounding boxes and skeleton keypoints
• A tracker module (ByteTrack) to give the detected people a unique ID so that you can count individual people’s exercise reps
• A matcher module to increase the accuracy of the tracker results, by matching people over frames so as not to lose their IDs
• CV2 visualisation to visualise the results of the detections and see the results of the application

And all of this in real time, on the edge, while the IMX500 is taking care of the AI inference!

Now both you and the AI Camera are testing out each other’s limits. How many pull-ups can you do?

We hope by this point you’re curious to explore further; you can discover more sample applications on GitHub.

The post Bringing real-time edge AI applications to developers appeared first on Raspberry Pi.

A few months back, we quietly dropped the Pico VS Code project on GitHub. It didn’t take long before the feedback started pouring in. Since then, we’ve been listening and tweaking. Now, we’re excited to officially unveil the public beta of the Raspberry Pi Pico Visual Studio Code Extension!

What is Pico VS Code?

Pico VS Code is a Microsoft Visual Studio Code extension designed to make your life easier when creating, developing, and debugging projects for Raspberry Pi Pico-series boards. Whether you’re a total beginner or a seasoned pro, this tool is here to help you dive into Pico development with confidence and ease.

If you’ve ever tried to set up an embedded development environment, you know it’s no small feat. Beginners often find themselves tangled up in the complexities of build systems, SDKs, and toolchains. And let’s not even get started on cross-compilation; developing on one machine to run code on another introduces a whole new set of challenges.

Getting all the right configurations and installations in place can be intimidating for everyone, not just those new to the game. Even experienced developers can find themselves tangled in frustrating setup processes that eat into valuable development time.

That’s why we created the Pico Visual Studio Code extension: a user-friendly tool that simplifies the entire development process. We wanted to offer something that takes the guesswork out of setting up your environment, so you can start coding in an interface you’re already familiar with — Visual Studio Code — as quickly as possible.

With Pico VS Code, you won’t have to worry about nitty-gritty details that trip up newcomers and sometimes stymie veterans. Instead, you’ll be able to focus on what really matters: bringing your Raspberry Pi Pico projects to life. Whether you’re working on your first blinking LED or a more complex project, Pico VSCode is there to help you get started and keep you moving forward.

How do I get Pico VS Code?

Prerequisites

To get started with the Pico VS Code extension, you’ll need to ensure that your development environment meets a few basic requirements. The extension is compatible with various platforms, including Raspberry Pi OS, Windows, macOS, and Linux, each with its own set of prerequisites.

All platforms require an install of Visual Studio Code version 1.92.1 or newer. For detailed instructions on setting up Pico VS Code on your platform, refer to the respective prerequisites outlined below.

Raspberry Pi OS

Windows

• Make sure you’re using a x86-based PC (not ARM64).

macOS

• For macOS users, you can install all necessary dependencies by running the following command in your terminal:

xcode-select --install

Linux

• Refer to the README.md on our GitHub page for a full list of required software. Many distros include the required software in the standard OS install.

Installing the Pico VS Code extension

You can install the extension in two ways. The first option is to install it directly from your editor’s marketplace. This provides a seamless integration and automatic updates. The second option is to download and manually install the package. This gives you more control and is helpful in restricted environments or when managing specific versions.

Install in your editor

You can download the Pico VS Code extension directly from the Visual Studio Code Marketplace within your editor.

If you’re using a different VS Code-compatible editor, the extension is also available through the OpenVSX marketplace.

Manual installation

If you prefer, you can manually install the extension by downloading the package directly. Follow these steps:

1. Visit the pico-vscode GitHub page and download the latest .vsix file from the release assets.
2. To install the file in VS Code:
   • Open the “Extensions” panel as described earlier.
   • Click the three-dots menu (...) above the search bar and select “Install from VSIX…”.
   • Choose the .vsix file you just downloaded.

Alternatively, you can install the .vsix file via the terminal using the following command:

code --install-extension raspberry-pi-pico-<version>.vsix

Building a blink example with Pico VS Code

To create a project based on a blink example, select “New Project From Example” in the Pico sidebar panel added by the extension. Then, search for the example you want to use — in our case, “blink” — in the project name field. Click the “Create” button to generate a project from the template.

Once the project is generated, it will automatically open. When a Pico project is opened, the extension configures the build system for you based on the SDK version, board type, and other settings you’ve selected. After the progress bar disappears, you can compile the project by clicking the “Compile” button in the bottom right corner. This will open an output panel where you can follow the build progress. Once completed, you should see a line like this: [61/61] Linking CXX executable blink.elf.

When it comes to uploading firmware to your Raspberry Pi Pico board, you have two options. Both require you to connect your Pico in BOOTSEL mode. To put your Pico into BOOTSEL mode, connect your Pico board to your host computer while holding the BOOTSEL button.

The easiest way to get your firmware up and running is the “Run” button. Connect your Pico board in BOOTSEL mode, hit the “Run” button, and the firmware will upload automatically. You’ll know it’s working when the tiny LED near the USB connector starts blinking as the board disconnects itself.

In addition to the blink.elf executable, you’ll find a blink.uf2 file in the build directory within your project. If you prefer to manually flash firmware, drag and drop this UF2 file onto your Pico board in BOOTSEL mode.¹ Once the file has been copied onto the board, it will automatically dismount and start running the blink project. You can confirm this by observing the tiny LED near the USB connector. It should start blinking as soon as the board disconnects itself.

For debugging directly within VS Code, the extension adds configuration to your project that allow you to use the debug panel and set breakpoints, just as you would when debugging other C/C++ projects on your computer. To run in debug mode with a Debug Probe, press F5. For detailed instructions on how to correctly wire your board for debugging, refer to the Getting Started guide on our website.²

Integrated offline documentation

When developing bare-metal code for the Pico, you often need to reference an API or check hardware specifications. To streamline your workflow, we’ve integrated the documentation directly into VS Code. This allows you to quickly access the information you need without leaving your editor — or requiring an internet connection.

To access the documentation, navigate to the Raspberry Pi Pico panel in your sidebar and select the topic you want to explore. The documentation will open within the editor, so you can position it wherever you need, just like any other file. This way, you can keep your code and reference materials side by side.

How to update the project configuration

A new SDK revision has been released, and you want to take advantage of its awesome new features in your Pico project. No problem — the extension has you covered. If you need to switch the target board, change the selected SDK, or adjust any other properties configured during project creation, the extension provides commands to update your project settings in just a few clicks.

For example, to switch the SDK version, you’ll find a UI element in the status bar at the bottom of the VS Code window, or in the Raspberry Pi Pico Project quick access panel in your sidebar, displaying your currently selected version. Clicking this will open a simple dialog where you can choose the new SDK version you want to use. Once selected, the extension will automatically reconfigure your project to use the new version. For optimal IntelliSense functionality, we recommended to reload your window after changing any project settings to ensure all extensions are aware of the new configuration. After the changes have been applied, the extension sends you a notification with a button that will reload the current window.

If you need to update settings other than the SDK version or board type, you can access additional commands via the VS Code command palette, which you can open with the keyboard shortcut Ctrl+Shift+P (or Cmd+Shift+P on macOS). Type “Raspberry Pi Pico,” and you’ll see a list of all available commands provided by the Pico extension. This makes it easy to adjust your project configuration as needed.

It also supports MicroPython

For beginners or developers who want to get their projects up and running on a Pico as quickly as possible, MicroPython is an excellent choice.³ It is a lean and efficient implementation of the Python 3 programming language, specifically designed to run on microcontrollers and in constrained environments. It includes a small subset of the Python standard library, making it a powerful yet lightweight option for embedded development.⁴

To create a Pico project using MicroPython instead of C/C++, select “New MicroPython Project”. You can find this button either in our Quick Access panel, located in your sidebar or by running the New Pico Project command and selecting MicroPython as language. This will launch the familiar project creation wizard, now tailored for setting up a MicroPython project. Choose the location for your project folder and set a name for your project. When you click “Create,” the extension generates the new project and opens it for you, just like with a C/C++ project. But instead of using C/C++, your new project uses the MicroPico extension to run your code on the board and manage project configurations.

With MicroPython, you can quickly start prototyping and experimenting with your Raspberry Pi Pico-series device, making it an ideal option for both newcomers and seasoned developers alike.

Next steps

For more detailed information on using the Pico VS Code extension, including a comprehensive list of settings and additional guidance, visit our GitHub project page. It’s a great resource for getting the most out of the extension.

If you’re new to developing Pico projects, don’t forget to check out the Getting Started guide we mentioned earlier—it’s packed with helpful tips to get you up and running.

If you’re looking to create a project that leverages the advanced features of Pico-series devices — such as I2C, PIO, or enabling stdio support—be sure to explore the “New C/C++ Project” interface. This tool allows you to customize your project setup to suit your needs, so you can dive into development quickly and efficiently.

1. https://www.raspberrypi.com/documentation/microcontrollers/c_sdk.html#your-first-binaries ↩︎
2. https://www.raspberrypi.com/news/raspberry-pi-debug-probe-a-plug-and-play-debug-kit-for-12/ ↩︎
3. https://www.raspberrypi.com/documentation/microcontrollers/micropython.html#what-is-micropython ↩︎
4. https://micropython.org ↩︎

The post Boost Your Pico Projects with the new Pico VS Code Extension appeared first on Raspberry Pi.