Author: cyruslab

[python]Read from delimited files and excel and convert to html with Pandas

Thanks to the creator of Pandas, data extraction from excel and delimited text file is made easy. Below is my modified function that suits to be used to extract data from delimited text files and excel files, then they are converted to html. HTML

[python]Data processing with Pandas

I used to use the with context and the open function to open and save data into file, however the module Pandas truly amazed me... It has methods like read_csv, read_excel which reads from csv and excel files, and methods like to_csv which export the data to csv file. How it looks like This file … Continue reading [python]Data processing with Pandas →

[python]Download multiple files from different directories from ftp server

File patterns This is to download multiple files of the same pattern form different directory. In ftp.pyclass.com there are directories organized in year, and within each year there are gunzip files that has this pattern stationId-Year on this example I will be downloading the gunzip files which has a station id 010010-99999, between the year … Continue reading [python]Download multiple files from different directories from ftp server →

[python]Download files with FTP client with multithreading

You may wish to enroll the course by Ardit Sulce, I will not reveal the username and password on his ftp.pyclass.com, to be fair to him in order to use his resource for practice and learning data science you got to enroll to his course. The code presented here has 5 parts: security.crypto, for encrypting … Continue reading [python]Download files with FTP client with multithreading →

[python]Comparing executed time between with and without threading

To compare between with and without threading, there are three routers from R1 to R3, the task is to save the show version on each router to all.txt. Do the tasks without threading The script runs sequentially on each device, and the total time is 15seconds. Do the tasks with threading The time taken to … Continue reading [python]Comparing executed time between with and without threading →

[python]Threading, acquiring and releasing lock to access shared resource.

On previous post about threading, there is no worry of data corruption as each fetch and write is to different destination and write to different file respectively. What if the requirement is to append all data fetched from multiple websites and write them into a single file? There will be race condition which the same … Continue reading [python]Threading, acquiring and releasing lock to access shared resource. →