[python]Delete many by objectid using pymongo

I have multiple duplicated entries, I think this is the problem with NoSQL type of database, although easy to use it has no normalization method like the relational type database. So my mongo database have duplicated like this: These are the duplicates when I accidentally run the script multiple times with the same device info. … Continue reading [python]Delete many by objectid using pymongo →

[selenium] Select option from dropdown box

There are two ways to do this, first find the xpath of the element of the option then issue a click method, but this method will not work on some website which monitors mouse hover event and enable the element if the mouse is hovered over the drop down box. Another sure way of selecting … Continue reading [selenium] Select option from dropdown box →

[python]Change interesting contents of a text file into dictionary

Introduction This exercise downloads the CHECKSUM file from the centos download page, the file downloaded is a temporary file and is destroyed once it is closed after used. The data read from the temporary file is then processed into a dictionary, the dictionary key is the filename and the value is the hash digest. Tempfile … Continue reading [python]Change interesting contents of a text file into dictionary →

[python]Download file and use tqdm for progress bar.

Download file with requests To download the file from web with requests module is the easiest, you just need to turn on stream while using get method. Then download the file chunk by chunk by using the iter_content method. Here is the example code on how to download file with requests. import requests from pathlib … Continue reading [python]Download file and use tqdm for progress bar. →

[python]Download geckodriver for windows

I am writing a python script that uses requests module to download the latest release of geckodriver.exe for windows 64-bit, I added a few functions to make the download slightly more dynamic by identifying the OS type and the system architecture, and after downloaded unzip the package into the defined path I specified. The current … Continue reading [python]Download geckodriver for windows →

[selenium] Handling authentication popup

This exercise will present a popup dialogue asking for your credential (which is admin, admin as shown in the exercise). This popup is not an alert nor an iframe, the only way is to include the username and password with the address to get through it with selenium. This is the original state after the … Continue reading [selenium] Handling authentication popup →

[selenium] Simulate right click.

Selenium works on web elements, so any browser's function selenium cannot automate. On this exercise, right click within the dotted box area will trigger a JS alert which says "You selected a context menu". To simulate right click with Selenium: First find the element of the dotted box, then use ActionChains to move to the … Continue reading [selenium] Simulate right click. →

[selenium] Dropdown box selection

Selenium is about finding the elements to do action upon, the dropdown box exercise can be found here. From the web developer tool of your selected browser find the element of option1 and option2 and copy their xpaths, xpath is the most accurate for selenium to act upon, my preference is always xpath then next … Continue reading [selenium] Dropdown box selection →

[selenium] Solution to HTML5 drag and drop with python

Selenium is a great web app automation tool which have several variants, I am using python to implement selenium on this https://the-internet.herokuapp.com/drag_and_drop, Selenium has an ActionChain that can do drag and drop, but this is not working in HTML5, the solution in another language (perhaps is ruby?) can be found here. I have converted the … Continue reading [selenium] Solution to HTML5 drag and drop with python →

[python] Adding extension to geckodriver with selenium

Webdriver I am using firefox hence I am downloading geckodriver to work with selenium. To set up the webdriver in the code do this: from selenium import webdriver driver_path = r"E:\webdriver\firefox\geckodriver.exe" driver = webdriver.Firefox(executable_path=driver_path) Install extension To install an extension to the webdriver for each instance do this: buster = r"E:\webdriver\firefox\buster_captcha_solver_for_humans-1.0.1-an+fx.xpi" driver.install_addon(buster, temporary=True) On every … Continue reading [python] Adding extension to geckodriver with selenium →