python webinput automation GUI with Selenium and PysimpleGUI

Project Description#

Simple python desktop application for automating repetitive web-based input tasks which load the data from a spreadsheet. developed using selenium as webdriver and pysimplegui as GUI.

GitHub link Video demo

Directory structure#

The main directory structure of the project. The structure is inspired by the mix of MVC (Model-View-Controller) pattern and DDD (Domain-driven design) pattern. I will briefly explain each directory and file below.

├───📁 gui (data and logic related to the GUI app)
├───📁 modules (main business logic for each domain separated by folder)
│   └───📁 example
├───📁 resource (various miscellaneous files like favicon, webdriver, etc.)
├───📁 utils (various utilities related to the app.)
│   ├───📁 auth
│   └───📁 webdriver
│   └───📄 crypt.py
│   └───📄 excel.py
│   └───📄 helper.py
├─📄 controller.py (the flow control logic between main business and route)
├─📄 route.py (event routes by main GUI to handle user action)
├─📄 auth.py (handle logic related to authentication)
├─📄 setting.py (handle logic related to user setting.)
├─📄 main.py (main wrapper connecting all the components to execute the app.)

Automation workflow #

1. The user enters the app with the specified number of rows of data.

2. when the app starts it will get the first row with the status "NONE" (will automatically add column status with value NONE if the excel sheet doesn't have one)

3. Each row on the GUI will be updated based on the outcome after the automation is completed.

4. Excel sheet row will be updated based on the outcome after the automation is completed.

4. If the automation "FAILED", the excel sheet will be updated with the reasoning why the automation failed (ex: username is invalid, the user doesn't exist, etc.)

5. The automation continues until there are no more rows with the status "NONE" present.

Afterword#

I have always had a fascination towards automation, when I was in the university I made simple automation using OCR/Image recognition for a game I played back then to automate mundane & boring stuff. Fast forward I got an opportunity to solve repetitive tasks problem which can be very time-consuming if the task is performed manually.

I started using the same technology I used since I am somewhat familiar with the requirement, which is using OpenCV for OCR/Image recognition. Although it works great on my PC, problems start to appear when a user has different resolution to mine which ended up the app not finding the image. Not only that, the app was resource-intensive as it needs to scan the whole desktop each time the automation moved. The speed also greatly depends on the user's computer specification.

I refactor everything in favor to use selenium, as it targets the browser DOM directly instead of relying on images. In addition to being lightweight, it is also much simpler to wire up to a functioning condition. While the automation itself works fine, I found a new problem I had to solve which is a way to make the app user-friendly as starting the script through a code editor is not very intuitive. After a little bit of research, I came across PySimpleGUI, the library's abundant examples and active maintainers were huge assistance to me.

During the process, I had to deal with a variety of problems, such as authentication, project architecture, compiling the app into an executable, and many other issues. In the end, I managed to finish the project in working order. The app was being used by multiple persons, as it improved the task completion speed significantly.

Demo Example#

Although I can't show the actual project I made as it's confidential, I am able to make a simple example repository of how the app functions. I turned off authentication if you are interested to test the demo on your own. using the mock data I made to target this sample practice website. you can see the full video demo here.