@mikael WOW with that package the speed increase a lot, thanks, now I know the real power of async programming

@mikael I try to implement the async function, really I don’t know if i do it well, now it download all more faster than before and in other hand i also correct the link extraction function because some times (most of the time ☹️) the function only outputs 20-50 urls, now it can extract all or some number near to all of the links in the page, I also make a setup.py file but I truly don’t know if it works.

Now the way to use it is:

from scrapthor import scrap
url=“some url”
scrap(url)

Please. can you check it?

I made this python script for web scraping, it can map a web site in a dictionary for example:

{url:[url_type,{url1_in_url:[url1_in_url_type,{...}],url2_inurl:[...],...}]}

if someone also want to download all files in that web_site only need specify it:
with the variable 'descargar' and set it to True

 url = 'https://en.wikipedia.org/wiki/Roy_Clark'
 descargar = True
 profundidad = 2
 archivo = 'clark.json'
 s = Scraper()
 s.lineal(url,profundidad,descargar,archivo)

GitHub

I want to implement some threading algorithm to speed up the analysis.
Note:

it sometimes have an error when downloading files.

@mikael You are right, FTP is completely insecure, and yes paramiko is perfect for secure communication, but I am doing a FTP Server from scratch because I want to understand how it works inside

Hi Everyone, I know that the ftp commands are link text but how the communication actually work, for example when I want to request the content of a folder how the cliente send it for example:

Client send: “NLST <folder_name>“
Server response: <folder_content>

Is it the correct format to send a ftp command?

I make and ftp server, but it only works with my own ftp client because I don’t know how the commands have to be send.

And with what socket family it work?

WOW that repository is amazing thank you so much, I am trying to implement a regression tree classification, you are right python was not made for massive ML :(

You are right, tonight I am gonna to publish it in github, I want to implement some kind of regression machine learning algorithm to the app, really i don’t know much about AI, all the machine learning I know is for scikit-learn, so program a real regression network from scratch is more slower than I was thinking.

This days I was working in a python personal trainer inspired in the post of other user @mikael - his awesome personal trainer app i made something near to his idea but a bit different, in my app you can add more exercise or delete the new ones, also you have local routines that cannot be delete, in the home menu you can make plots of how your develop was... this is work in progress, I hope you can enhance it with ideas that I don’t have, actually I was working In some kind of machine learning for the app, to know how the user evolve in the exercises.

I hope you like it. Please comment any idea

You need all the files in the folder and sorry but my natal lenguage is Spanish so the app is in SPANISH

my app (google drive)

I made this clone of the AppStore app Mindnotes

I hope you like it

Mindnotes app clone