Building Custom Apps for the Rokid Smart AI Glasses: What Works and What Doesn't
イチケン / ICHIKEN
In this video, maker and YouTuber Ichiken tries the Rokid Smart AI Glasses for the first time. The question is how far a pair of consumer smart glasses can go, both with the apps that ship on the device and with apps you write yourself. The first half covers the built-in features, comfort and practical limits. The second half shows three custom apps built with help from Claude. The speaker's overall view is that the glasses are already practical for some tasks, especially translation. Real-time AI assistance, however, is still limited by latency and on-device processing power.
The Hardware and First Impressions
The Rokid Smart AI Glasses look like a slightly chunky pair of glasses. A closer look shows a speaker near the temples, control buttons on the frame, and a forward-facing camera. The lens area doubles as a display that shows text. What caught the speaker's interest was that, beyond the pre-installed apps, they could write their own app and run it on the glasses.
This was the speaker's first time using any smart glasses. You open menus and features by tapping or swiping on the frame. The AR display appears to sit about five meters in front of you. The speaker found it quite hard to focus on the display at first. Their trick was to cover the glasses with a hand so the background disappears, which makes the AR text easier to lock onto. They concluded it is mostly a matter of getting used to it.
Real-Time Translation
Of the built-in features, translation and subtitles were the speaker's favorites. They were surprised by how accurate real-time translation was. They see it as a powerful tool for face-to-face conversations in particular.
For the demo, the speaker spoke Japanese to an AI that answered in English, as a stand-in for a foreign-language conversation partner. The speaker asked about hobbies. The AI replied that it enjoys exploring ideas, learning languages and conversations like this one. In the Japanese display, part of the answer came out as something like "hub," which the speaker pointed out. Next, the speaker asked what the AI would like to drink. The exchange went back and forth smoothly: the speaker said they would have tea, and the AI asked what kind. The translation kept up nearly in real time, and the speaker called it "very well made."
To test longer speech, the speaker asked the AI to say something lengthy in English. The AI gave a short reflection on how small daily routines, such as making a hot drink or stepping outside for fresh air, can set the rhythm for the rest of the day. The translation was fairly accurate. The speaker noted that the audio coming through the glasses was not ideal and that things did not always go perfectly. Even so, they judged the result to be at a level that is "quite practical."
The speaker argued that translation today is far more accurate than it used to be. If both people wore smart glasses, they think a conversation could feel quite natural. The same thing is possible with a smartphone, but they think the glasses look more natural in conversation.
Subtitles (Live Transcription)
The subtitle feature is plain transcription with no translation. The speaker also described it as a feature they really like. To test it, they played a clip from one of their own videos, which explains that the Raspberry Pi RP2040 microcontroller needs a 3.3V supply and cannot run directly on the 5V from a USB port, so a voltage regulator steps the voltage down. The transcription was quite accurate. As a side note, the speaker suggested the feature could help people whose hearing has started to decline.
Camera and Navigation
The camera sits in the frame, so you can take photos and videos without pulling out a phone. A button on the top of the frame controls it:
- Press once to take a photo.
- Press and hold for about two seconds to start recording video.
The speaker called the image quality standard rather than impressive. They also warned that using the camera at the wrong moment could cause trouble, so users need to be careful about when they use it.
The glasses also have a Navigation Mode. The speaker does not use it because they find walking while using it a little dangerous. They allowed that it might become convenient with practice. For now, though, a smartphone shows more information, so they think phones still have the edge for navigation.
Wearing Them: Prescriptions, Weight and Field of View
People with good eyesight or contact lenses can wear the glasses as they are. People who wear glasses cannot put the smart glasses on over their regular pair. They need either contact lenses or prescription lenses, which are sold separately. According to the speaker, you can order these on the official website starting at about 15,000 yen, and they attach magnetically over the original lenses. The timing did not allow the speaker to get prescription lenses before filming, so they used contact lenses.
The glasses are a bit heavy. The speaker's regular glasses weigh 19 g, and the Rokid glasses weigh 52 g. They are clearly heavier than normal eyewear. Given everything packed inside, though, the speaker was impressed that Rokid made them this light. They think making them much lighter may be technically difficult but still hope for it.
The bigger complaint was the field of view. The frames are thick, and there is some distance between the eyes and the lenses, so the glasses noticeably block your view. That is acceptable in daily life. For driving, however, the speaker thinks the blind spots are significant enough that you should not wear them. The speaker normally chooses frames that stay out of their field of view entirely, so the Rokid glasses felt quite out of place to them. They hope future versions improve the field of view and reduce blind spots.
Battery Life
The speaker said battery life is not good. The battery holds only 210 mAh, so if you keep an app running constantly, you will need to charge several times a day. Custom apps that are not optimized for power, like the ones shown later, drain the battery especially quickly. On the plus side, the battery charges to 80% in 20 minutes. The speaker also mentioned that some compact batteries apparently let you use the glasses while charging, and suggested trying one.
Developing Apps: YodaOS and Vibe Coding
The glasses run YodaOS, an operating system Rokid has optimized for smart glasses. Because YodaOS is based on Android, apps you build yourself can run on it. The speaker got a separate developer cable to connect the glasses for development.
App development normally requires tools like Android Studio. The speaker handed the entire process to Claude, starting with setting up the development environment. They described this as "vibe coding." They built three apps as demos.
App 1: Hello Rokid
The first app simply displays "Hello Rokid," to confirm that a self-built app runs on the glasses. It worked without problems.
App 2: Math Solver
The second app uses the Rokid camera to photograph problems on paper or a screen and then solves them. With the problems in view, you tap the frame to take a photo, then tap again to have the AI analyze the image. The display lists the problems it found. You swipe to choose one and tap to select it, and the AI solves it and shows the intermediate steps.
The first example was the integral of x to the fifth power, and the speaker confirmed the answer was correct. The app also solved the other problems on the page correctly. The last one got what the speaker called a "clear and thorough answer." The speaker then tried definite integrals, where the interval is specified. The app read those from photos without trouble, and the answers were again correct.
The app works by sending the camera image over Wi-Fi to an external AI, in this case Claude's API, which reads and solves the problem. The speaker's point was that processing photos and videos with an AI outside the glasses greatly widens what the glasses can do. They suggested that if the camera captures a clear image and the AI understands it, the same approach could handle physics, English or other subjects.
App 3: Debate Assistant
The third app targets a use case where real-time response matters more. The glasses' microphone transcribes the conversation live. An external AI analyzes the transcript and suggests ways to rebut the other side, and the suggestions appear on the AR display. Since the other person cannot see the display, you can argue with AI help and gain an edge.
The debate topic was a classic Japanese snack rivalry: which is the better treat, Kinoko no Yama or Takenoko no Sato. An AI opponent took Kinoko no Yama, and the speaker defended Takenoko no Sato. The opponent argued that Kinoko no Yama comes in bite-sized pieces that are easy to pick up, that the chocolate and cracker are well balanced, and that it is easy to eat with one hand.
The speaker countered that Takenoko no Sato is easier to hold because its wide base makes it stable. The opponent replied that the mushroom's stem fits snugly around the finger and that its solid base lets it stand upright on a desk. The speaker then argued that the essence of sweets is enjoying the taste, and asked whether the conversation had drifted off topic into ease of eating. The opponent called this "a really sharp observation" and conceded that taste and texture matter most, before turning to the different ways people enjoy sweets.
The speaker judged that the app produced rebuttal strategies fairly well. However, there was a lag of a few seconds between the speech, the transcription and the suggested response, so they would not call it very practical yet. The delay comes from doing both transcription and rebuttal generation outside the glasses.
The speaker explained that transcription could run locally on the glasses. However, limited RAM means only a small model fits, and in their experience its accuracy was so poor that the results were "a total disaster." That is why the demo also sent transcription to an external AI service. The speaker expects that once a low-power, high-performance dedicated chip with an NPU becomes available, processing could happen quickly and more interesting apps would become possible.
Conclusions
The speaker had assumed smart glasses were purely consumer products. Being able to build custom apps changed that view. With apps tailored to specific tasks, they think smart glasses could also fit business settings such as manufacturing. On the practical side, real-time translation turned out to be genuinely useful, and they plan to use it on their next trip abroad. The speaker expects smart glasses as a gadget category to keep evolving over the next few years and intends to keep following it.
Hello. Since I got these smart glasses today, I'd like to try out a variety of things. Here's what it looks like in real life. This is the Rokid Smart AI Glass. They may look like a bit of a chunky pair of glasses, but if you look closely, you'll see that there's a speaker in this area, control buttons, and a camera in front of you. What's more, the lens area serves as a display, and text will be displayed.
What's interesting about these smart glasses is that, in addition to the apps that come pre-installed, I was able to develop the app all by myself and actually run it on these smart glasses. In the first half of the video, we'll introduce the basic features, and in the second half, we'll actually try our hand at app development. Just how much can we actually do? Let me introduce them to you.
But first, let me introduce our sponsor. If you want to make printed circuit boards on a budget, we recommend JLCPCB. Lately, I've been really focusing on manufacturing multilayer printed circuit boards, and they really want everyone to try using 6-layer PCBs. Five pieces of 6-layer 100 × 100 mm circuit boards for $36.80. Pad-on-Via, which is essential for high-density designs, has recently become available for free. We're also distributing coupons for 6-layer PCBs, so please feel free to try out all sorts of things yourselves.
Here's what it looks like when I wear the smart glasses. This is my first time using smart glasses or anything like that. When you put on the smart glasses, the display looks like this. Tap on the frame or swipe, and various menus appear. You'll be able to do all sorts of things.
I'm also showing the screen that's currently visible, but it is in focus about 5 meters ahead. At first, it was pretty hard to focus on this AR. But cover the smart glasses with your hand like this. If you make it so you can't see anything, it makes it easier to focus on the AR portion. It's just a matter of getting used to it.
Now let's actually try using it. It has a variety of uses, but I'll be introducing only the ones I particularly liked. First of all, my favorites are translations and subtitles. The translation happens in real time, and I was surprised at how accurate this was. Especially when talking to people face-to-face, for example, it will be a very powerful tool.
Now let's try using it. Select a translation here. It is currently being translated into Japanese. For the AR section, the Japanese text is displayed correctly. This is simply a transcript. For example, if the other person speaks English, I'll try this assuming I'm speaking in Japanese. The person you're talking to is an AI.
AI, let's have a quick conversation. What are your hobbies?
If I had hobbies, I'd say things like exploring ideas, learning languages, and conversations like this.
It's turned into a “hub,” hasn't it? Let's have a quick conversation. What would you like to drink?
I'm fine with water, but if you're the one choosing, what would you like to drink?
I'm going to have some tea.
Nice! Tea. Any particular kind?
It looks something like this. The conversation is flowing smoothly, and this translation is also being done pretty much in real time. It's very well made. I'd actually like to try writing a slightly longer piece. Could you please say something in English that's a bit long?
Certainly, here is a longer passage in English. When I think about daily life, I realize that even the smallest routines often shape how the day feels. Something like making a hot drink, stepping outside for a breath of fresh air, or just taking a quiet breath. If you start just one small ritual, the rest of the day can follow that rhythm.
A pretty accurate translation like this. I do wonder if I'm doing it right. The sound quality coming through the smart glasses is... The translation isn't very good, but things don't always go as planned. Even so, it's quite practical. I guess you could say it's at an acceptable level. I think it's at a level that's quite practical.
Translations these days are different from those of the past. Since the accuracy has improved quite a bit, if the other person is also using smart glasses, I think we'll be able to have a conversation that feels quite natural. It's not that you can't do it with a smartphone, but it's better to use smart glasses. It does look more natural, doesn't it?
As for the subtitles, it's a feature I really like. This is it, isn't it? For example, if you try playing a video.
Let's take a look at the other parts as well. The microcontroller I used this time is the Raspberry Pi RP2040. This microcontroller itself requires a 3.3 V power supply, and it does not operate on the 5 V power supplied by the USB port. So, using a voltage regulator, I need to step down the voltage from 5 V to 3.3 V.
Do you understand? The transcription is pretty accurate, isn't it? For example, if your hearing starts to get a little worse, in such cases—this is just a side note—I think this feature could be quite useful.
Next up is the camera feature. Since the camera is built in here, you can capture photos and videos instantly without even taking out your smartphone. There's a button right at the top of the frame. Press this button once to take a photo, and if you press and hold this for about 2 seconds, video recording will begin.
This is what the video actually looks like. It's not exactly stunning, but standard image quality. However, when it comes to the camera features, if you don't choose the right time to use it, this could lead to problems. You need to be careful about that.
There's also something called “Navigation Mode.” Since it can be a little dangerous to walk while using this, I don't ever use it. I suppose it might be convenient once I get used to it. Besides, smartphones simply have more information, so as things stand, smartphones have the edge. That concludes our introduction to the apps that come standard with the device.
How they actually feel to use and how comfortable they are to wear: let's take a closer look at that. First, for people with good eyesight or those who wear contact lenses, you can wear these glasses just as they are. For people who wear glasses, wearing these smart glasses over the glasses you normally wear is not possible. So, you will need to wear contact lenses or purchase prescription lenses, sold separately.
Regarding prescription lenses, you can create them on the official website, starting at about 15,000 yen. These are prescription lenses that attach using magnets on top of the original lens. In Ichiken's case, since we weren't able to make prescription lenses at the time, for now, I'm using contact lenses.
Here's what it sounds like when you actually play it. It's a little on the heavy side. The glasses I usually wear weigh 19 grams. The Rokid model weighs 52 g. They're heavier than regular glasses, but with the various things included inside the smart glasses, I can't believe they managed to make it this lightweight. That's the assessment.
As for the change in your field of vision when you put them on, the frames are thick, and since there's a bit of a gap between my eyes and the lens, it really obstructs your view. It's fine for everyday use, though. If you're driving a car or something, since this creates quite a few blind spots, I think it's better not to use it.
My criteria for choosing eyeglass frames are that no frames or other objects appear in the field of view. In other words, I'm choosing them based on the criterion that they don't obstruct my view, so when people like that put on Rokid's smart glasses, it feels pretty out of place. This can result in a wide field of view or blind spots. As for those points, I hope they'll make improvements in the future. As for the weight, to make it even lighter than this, I do think it might be technically difficult, though. I'd like to keep my hopes up.
By the way, the battery life isn't very good. This is because the battery has a low capacity of 210 mAh. If you always have some kind of app running, you'll need to charge it several times a day. In particular, when running a custom app that isn't optimized for power consumption, such as the ones I'll introduce later, the battery drains pretty quickly.
Even if the battery runs out, you can charge the battery to 80% in 20 minutes. That's a silver lining. Also, it seems there are some compact batteries that can be used while charging, so I think it would be a good idea to try using those together as well.
Next, with these smart glasses, we will continue to develop useful apps. As for the OS in these smart glasses, it comes with an operating system called YodaOS that Rokid has optimized for smart glasses. However, since YodaOS itself is based on Android, you can also run the app I created for smart glasses in here. So, let's actually try making something.
To create an app, I needed a developer cable, so I've obtained that separately. For app development, you'll need a development environment with tools such as Android Studio, but I'll just leave it all to Claude, the AI. Starting with setting up the development environment, they took care of everything, including the lettering. That's what they call “vibe coding,” isn't it? Since I created three apps this time, partly as a demo, I'll introduce them to you.
First, to verify whether the app we built actually works on smart glasses, I created an app that displays “Hello Rokid.” Here it is. It's working just fine. I don't think there's any problem with this, at least for now.
Next, I tried making an app called a math solver. This uses the Rokid camera to view the problem visible on paper or a display. It's an app that solves problems for you. With the problem in view, tap the frame to take a photo. If you tap once more while in this state, the AI is analyzing the problem, isn't it?
As you can see on the screen right now, it shows which problems to solve. If you swipe this, you can choose a problem. When you select and tap that question, the AI starts solving it. For example, this is the integral of x to the power of 5. In that case, it now displays the answer and the intermediate steps. For the record, the answer to this problem is correct.
It looks something like this. I'm solving the problems correctly. Regarding this problem as well, it's completely solved. Let's try solving this last problem as well. A clear and thorough answer. You'll see that it's correct. It's pretty amazing, actually.
I just finished solving a problem on indefinite integrals, but problems where the integration interval is specified, such as this, you can take photos without any problems. It's reflected in the glass right now, and the answer is correct. I think it's pretty good, don't you?
As for how the current math solver worked, images taken with a camera are sent via a Wi-Fi network to an external AI—in this case, Claude. Send the problem to Claude's API to have it analyzed and solved. That's what I'm doing.
The point of this demo is that by processing the photos and videos you've taken using AI outside the glasses, the range of possibilities expands significantly. This time, I worked on some math problems, but if the camera can take clear photos and the AI can understand the content, then we can solve problems in subjects other than math, such as physics or English, too.
Next, I tried my hand at building an app where real-time performance is even more critical. This is an app called “Debate Assistant.” Since the smart glasses have a microphone, it transcribes conversations in real time. By having an external AI analyze the transcribed content, I'd like it to come up with a strategy for refuting the argument. The reply will be displayed on this glass using AR. Since the other person can't see it, you can refute your opponent's argument while gaining an edge with the help of AI.
With that said, I'd like to get started. The debate topic is Kinoko no Yama and Takenoko no Sato. The question is which one is the better treat. You're “Kinoko no Yama,” and I'm “Takenoko no Sato.” All right, let's get started. Please go ahead from there.
Got it. Kinoko no Yama comes in bite-sized pieces that are easy to pick up with your fingers. The balance between the chocolate and the crackers is just right, and its main appeal is how easy it is to eat with just one hand. The contrast between the chocolate part and the toasty aroma of the crackers is simple, yet very well-crafted. In other words, it's easy to hold and easy to eat. They say the flavors are well-balanced. As an everyday snack, I think it's very well done.
Takenoko no Sato is easier to hold. It has a wide base, so it's stable, right?
I understand why it might look that way, but actually, the stem of the mushroom fits snugly around your finger, and it feels stable when you hold it. Plus, the foundation is solid, so there's also the simple advantage that you can stand it upright on your desk. Something that's great to hold in your hand or set down. It's flexible, isn't it?
While the essence of sweets is to enjoy them as they are, aren't we getting off topic from the discussion about ease of eating?
That's a really sharp observation. It's true that the taste and texture are, after all, the most important aspects of sweets themselves. On the other hand, ways to enjoy sweets...
As we've just seen, whether it's a debate or strategies like refuting arguments after a conversation, I do think it's working pretty well. There's a time lag of about a few seconds between the transcription and my response, so it's hard to call it very practical. Regarding the transcription and the strategy for refuting arguments, since it's happening outside the smart glasses, that part just takes a lot of time.
Regarding the transcription section, this can also be handled through local processing on the glasses themselves. However, due to RAM capacity limitations, I couldn't use a larger model, and we have to use a smaller model. As a result, the transcription accuracy is quite low, and the results turn out to be a total disaster. What I was doing just now was outsourcing the transcription work to an external AI service as well.
In this regard, when a low-power, high-performance dedicated SoC equipped with an NPU becomes available, it can process data quickly, and I think we might be able to do something even more interesting. Similarly, when it comes to app development, I realized it can actually do quite a lot of things.
So, I went ahead and tried out Rokid's smart AI glasses. I had thought smart glasses were consumer products, but just like we tried this time, you can even develop apps. In that case, if we can develop an app tailored to a specific purpose, I thought it might be a good fit for the B2B sector, such as manufacturing, as well.
From a practical standpoint, since I've found that real-time translation is actually quite useful, the next time I go abroad, for example, I definitely want to make the most of this. As for smart glasses themselves, I think the gadget sector will continue to evolve over the next few years, so I'd like to keep following this going forward.
If you're interested, please check it out via the link in the description. If you found this video helpful, please give it a thumbs up. If you have any questions or concerns, please feel free to leave a comment as well. Thank you for watching the video all the way to the end.
Article published · Updated