Assisting Users to Fill Out Long Forms with Conversational AI
A revolutionary RPA solution that combines Conversational AI with Computer Vision to automate complex form filling for real estate agents. The system uses plain English conversations to guide users through lengthy forms while automatically filling them in real-time.
Unlike traditional RPAs that rely on selectors, our solution employs Computer Vision to understand GUI elements, providing a more robust and human-like interaction with back-office software.
This innovative approach allows the system to navigate complex software interfaces just like a human would, making it more adaptable and resilient to changes in the user interface. The chatbot can handle multiple screens, forms, and workflows seamlessly while providing real-time guidance and recommendations to users.
Watch how our RPA chatbot assists users in filling out complex forms
Our client is a real estate agency with 100+ agents. For every deal, they must fill out 10+ lengthy forms in their back-office software. This was time-consuming for busy agents. And even impossible for newbie agents. Many would abandon forms in the middle, or would fill them incorrectly or slowly! The aim is to increase the number of forms that are filled in, correctly.
On one side we would keep open the back office software, and on another side, we would keep open our chatbot. Using plain English, our chatbot would prompt the user with questions, & automatically fill the forms based on answers. Users could observe forms getting filled out on their screens. The chatbot would navigate across the sophisticated sequence of steps – by navigating between screens of software – without the user having to leave our chatbot. On a few certain confusing fields, the chatbot would also advise users on what to fill in those fields.
In 8 weeks, the team at SmarterCodes built an RPA + Chatbot, which our real estate agency client began to beta test across their team.
This RPA was built differently. Typical RPAs use the "selector" technique to identify GUI elements and operate on them. Ours rather employed Computer Vision to understand the GUI elements. This resulted in a better developer experience because some of the GUI elements in back office software weren't operable with the selector technique. But with Computer Vision we were able to operate just like a human agent would. To interpret the language of the user, we used a mix of Google Dialogflow and our own proprietary NLP AI. Google Dialogflow's ready-made AI helped us prototype faster, and our proprietary NLP AI helped us interpret some curve-ball natural language inputs.
Advanced RPA capabilities combined with Conversational AI
Cutting-edge technology stack for intelligent automation
Let's discuss how RPA and AI can streamline your operations and improve efficiency