Imagine turning your everyday clothes into smart motion-tracking tools. That’s exactly what a team of researchers at Cornell University has achieved with SeamPose. This innovative project, led by Catherine Tianhong Yu and Professor Cheng Zhang uses conductive threads sewn over the seams of a shirt to transform it into an upper-body pose-tracking device by unleashing the power of XIAO nRF52840. Unlike traditional sensor-laden garments that change the clothing’s appearance and comfort, SeamPose blends seamlessly into everyday wear without compromising aesthetics or fit.

This project offers exciting potential applications in areas like health monitoring, sports analytics, AR/VR, and human-robot interaction by making wearable tracking more accessible and comfortable.

Hardwares

To build SeamPose, the following key hardware elements were used:

• • Long-sleeve T-shirt with machine-sewn conductive thread along the seams.
  • Customized Sensing Board:
    • XIAO nRF52840 
    • A 36x31mm board with two FDC2214 capacitance-to-digital converters
    • 3.7V 290mAh LiPo battery

How SeamPose Works

SeamPose operates by transforming ordinary seams in a long-sleeve shirt into capacitive sensors, enabling real-time tracking of upper-body movements. Conductive threads, specifically insulated silver-plated nylon, are machine-sewn along key seams—such as the shoulders and sleeves—without altering the garment’s appearance or comfort. As the wearer moves, the seams stretch and shift, causing variations in their capacitance. These signals are captured by a customized sensing board integrated with two FDC2214 capacitance-to-digital converters and the XIAO nRF52840 microcontroller. The XIAO nRF52840 transmits the data wirelessly via Bluetooth Low Energy (BLE) to a nearby computer for processing.

The transmitted signals are fed into a deep learning model that maps the seam data to 3D joint positions relative to the pelvis. This allows the system to interpret complex upper-body movements using only eight sensors distributed symmetrically along the shirt. During testing, SeamPose achieved a mean per joint position error (MPJPE) of 6.0 cm, comparable to more invasive tracking systems. The XIAO nRF52840 ensures seamless real-time data transmission, making SeamPose a breakthrough in wearable technology by combining comfort with precise motion tracking.

Source: Cornell University Team

What’s Next for SeamPose?

SeamPose offers a glimpse into the future of smart clothing, where everyday garments become powerful tools for motion tracking without sacrificing comfort or design. However, challenges like real-world deployment considerations and smart-clothing manufacturing at scale need to be addressed for broader adoption. The research team plans to explore improved seam placement and enhanced sensor calibration for even more accurate tracking in the future.

If you’re excited about SeamPose and want to dive deeper, check out their paper on ACM Digital Library

MIT Media Lab researchers Cathy Mengying Fang, Patrick Chwalek, Quincy Kuang, and Pattie Maes have developed WatchThis, a groundbreaking wearable device that enables natural language interactions with real-world objects through simple pointing gestures. Cathy conceived the idea for WatchThis during a one-day hackathon in Shenzhen, organized as part of MIT Media Lab’s “Research at Scale” initiative. Organized by Cedric Honnet and hosted by Southern University of Science and Technology and Seeed Studio, the hackathon provided the perfect setting to prototype this innovative device using components from the Seeed Studio XIAO ESP32S3 suite. By integrating Vision-Language Models (VLMs) with a compact wrist-worn device, WatchThis allows users to ask questions about their surroundings in real-time, making contextual queries as intuitive as pointing and asking.

Credit: Cathy Fang

Hardwares

The WatchThis project utilizes the following hardware components:

• • Seeed Studio XIAO ESP32S3 Sense (with camera expansion board)
  • Seeed Studio round display for XIAO
  • OV2640 camera with long ribbon cable
  • LiPo Battery
  • Custom 3D printed parts

How the Project Works

WatchThis is designed to seamlessly integrate natural, gesture-based interaction into daily life. The wearable device consists of a watch with a rotating, flip-up camera attached to the back of a display. When the user points at an object of interest, the camera captures the area, and the device processes contextual queries based on the user’s gesture.

The interaction begins when the user flips up the watch body to reveal the camera, which then captures the area where the finger points at. The watch’s display shows a live feed from the camera, allowing precise aiming. When the user touches the screen, the device captures the image and pauses the camera feed. The captured RGB image is then compressed into JPG format and converted to base64, after which an API request is made to query the image.

The device uses these API calls to interact with OpenAI’s GPT-4o model, which accepts both text and image inputs. This allows the user to ask questions such as “What is this?” or “Translate this,” and receive immediate responses. The text response is displayed on the screen, overlaid on the captured image. After the response is shown for 3 seconds, the screen returns to streaming the camera feed, ready for the next command.

The software driving WatchThis is written in Arduino-compatible C++ and runs directly on the device. It is optimized for quick and efficient performance, with an end-to-end response time of around 3 seconds. Instead of relying on voice recognition or text-to-speech—which can be error-prone and resource-intensive—the system uses direct text input for queries. Users can further personalize their interactions by modifying the default query prompt through an accompanying WebApp served on the device, allowing tailored actions such as identifying objects, translating text, or requesting instructions.

Applications

Imagine strolling through a city and pointing at a building to learn its history, or identifying an exotic plant in a botanical garden with a mere gesture.

The device goes beyond simple identification, offering practical applications like real-time translation of, for example, menu items, which is a game-changer for travelers and language learners alike.

The research team has discussed even more exciting potential applications:

• • A “Remember this” function could serve as a visual reminder system, potentially aiding those who need to take medication regularly.
  • For urban explorers, a “How do I get there” feature could provide intuitive, spatially-aware navigation by allowing users to point at distant landmarks.
  • A “Zoom in on that” capability could offer a closer look at far-off objects without disrupting the user’s activities.
  • Perhaps most intriguingly, a “Turn that off” function could allow users to control smart home devices with a combination of voice commands and gestures, seamlessly integrating with IoT ecosystems.

While some of these features are still in conceptual stages, they paint a picture of a future where our interactions with the world around us are more intuitive, informative, and effortless than ever before.

Build Your Own WatchThis

Interested in building your own WatchThis wearable? Explore the open-source hardware and software components on GitHub and start creating today! Check out their paper below for full details.

Hey community, we are excited to announce that three of our powerful XIAO ESP32 development boards are now fully integrated with Wokwi, the popular online electronics simulator! You can now experiment, test, prototype and build projects virtually with the XIAO ESP32C6, XIAO ESP32S3, and XIAO ESP32C3 directly from your browser. A big shout-out to Uri Shaked, Founder and Creator of Wokwi, for making this happen.

What is Wokwi?

Wokwi is an innovative online electronics simulator that supports a wide variety of microcontrollers, sensors, and peripherals. Whether you’re working with an Arduino, ESP32, STM32, or other popular boards, Wokwi gives you a fast and easy way to design and test embedded projects – all from the convenience of your browser.

Key Benefits of Wokwi:

• • Start instantly: No need for downloading complex software or waiting for hardware.
  • Learn by experimenting: Mistakes won’t fry your components – experiment freely without fear!
  • Debug with confidence: With Wokwi’s virtual logic analyzer, WiFi simulation, and GDB debugging, you can tackle both hardware and software challenges.
  • Share your work easily: A simple link to your Wokwi project is all you need to get feedback from the community.
  • Unlimited parts: No more scouring for hardware – use as many virtual components as you need without worrying about cost or availability.

Wokwi is free to use and perfect for beginners and experienced developers alike. For professional users, the Wokwi Club offers access to advanced features and lets you shape the platform’s future.

Integration of 3 XIAO ESP32 Dev Boards on Wokwi

This integration reaffirms our commitment to providing makers and developers with cutting-edge tools that enable rapid prototyping and seamless development. 

With the integration you can simulate your XIAO ESP32-based projects and start coding even without these boards at hand right away. It also allows developers to take full advantage of the flexibility of Wokwi to prototype IoT applications, debug projects, and refine ideas before working with real hardware.

Simulate, Experiment, and Share Your XIAO Projects

It’s time to bring your ideas to life! With the XIAO ESP32C6, XIAO ESP32S3, and XIAO ESP32C3 now available on Wokwi, you have the tools you need to simulate, prototype, and refine your projects – all in a virtual environment

If you think you made something cool, share your projects with us! We’d love to see what you create. Tag us on social media or post your Wokwi project links. Let’s build something amazing together and inspire others in the community.

BTW., these 3 dev boards are just the beginning. We’re working hard together with the team at Wokwi to make more Seeed products available on Wokwi. Feel free to comment to let us know which product you want to see live on Wokwi.

Tired of all those unused keys collecting dust on your keyboard? Meet MONOKEY, a one-button Bluetooth keyboard designed to simplify your tech experience. Created by Big Bad Cult, MONOKEY is a revolutionary wireless 1% mechanical keyboard that brings utility, fun, and personalization to your daily tasks. Whether you’re looking for a practical key charm, a fidget tool, or a programmable button for macros, MONOKEY offers endless possibilities—all powered by the reliable XIAO ESP32C3 microcontroller.

Hardwares

The MONOKEY project is built using the following hardware components:

• • XIAO ESP32C3
  • Hot-swappable key switches
  • USB-C charging port100 mAh battery
  • Custom keycaps (provided by TinyKeyboardShop)

How the Project Works

The MONOKEY is a compact and portable one-button Bluetooth keyboard that connects effortlessly to any Bluetooth-enabled device, including smartphones, tablets, and computers. Out-of-the-box Bluetooth connectivity allows it to pair quickly with any compatible device, making it instantly ready for use with just one app for mapping. You can program MONOKEY to perform a wide range of tasks, from executing complex keyboard combinations to controlling media playback or even simplifying repetitive tasks like typing passwords with a single press.

Thanks to its hot-swappable key switches, MONOKEY is customizable—you can easily swap key switches and keycaps to match your personal style and preferences without needing to solder. Additionally, MONOKEY supports Wi-Fi connectivity, enabling advanced users to develop custom firmware using Arduino libraries. This feature opens up endless possibilities for creating custom commands or macros.

MONOKEY is powered by a 100 mAh rechargeable battery, providing long-lasting use. Charging is made simple with a USB-C port, ensuring fast recharges when needed. Its lightweight and compact design make it highly portable, allowing you to carry it with ease and add an extra layer of personalization to your tech setup wherever you go.

Get Your MONOKEY Today!

Interested in simplifying your life, one key at a time? The MONOKEY has been launched on Kickstarter with over 400% funded. You can still order one  today for just $39! Shipping is available worldwide, with a $5 flat rate for the US and Canada. Get your MONOKEY before Christmas and revolutionize your workflow with this practical, customizable one-key Bluetooth keyboard. For more details and to join the community, hop onto their Discord. Click your worries away!