We hope that Ikigotchi Garden will help amateur gardeners keep their plants healthy and support greener living.
Ikigotchi Garden stores a user's plants as a species, image, and watering history locally, and allows the user to upload images to use Gemini to check the species of a plant, health status of a plant, and to generate pixelated images of their plants. Users can also upload images to check the genus of a plant using a custom fine-tuned model.
Ikigotchi Garden uses React native and Expo Go to easily run on both android and iOS, and can be easily distributed on android. We used the Gemini API to support our initial species and health checking and image generation, and used PyTorch to finetune a small Vision Transformer on data from PlantNet.
The initial MVP uses Expo Go which makes pushing notifications more difficult. We are also hosting our own AI for the first time, so it was not that easy. We also had difficulties finding data for our Vision Transformer, especially because there are so many species of plants - we could not hope to include all the plant species, both because of a general lack of data and because data was not particularly centralized.
We are proud to have made our own Vision Transformer, with Transformers being the hallmark of AI today. In addition, we are glad to contribute another open source to the community, if nothing else for the learning experiences of ourselves and others.
We had not made AI models on this scale before this project, and we hope that such experience will help us later in our careers. Also, some of us had not made full-scale projects like this before, and this kind of fullstack development is extremely valuable.
We hope that in the future we can implement additional homemade AI features, such as a fine tuned “small” language model with RAG from web scraping for health advice and a separate Vision Transformer to determine if plants are healthy or not. We would also like to implement more gamified features so that users feel more engaged with their plants. We could also set up a Postgresql database with Python to help retrieve and recognize plants from user images.
