For Bioinformatics Support

• loadcsv also support tsv format file (\t split type), just using:
  • loadcsv test.tsv tb1 1
• loadcsv also support bioinformatics .maf/.vcf/.sam/.gtf/.gff/.gpd file, just using:
  • loadcsv test.maf tb1 or loadcsv test.vcf tb1
• Load GenBank Format Features Data.
  • using: loadgb gbfile tb01
  • load bioinformatics GenBank file (etc: .gb/.gbff/.gpff) features info to table.

Crawl website data, parse and write to database

• using: loop loadweb url=https://xxx.xxx.com/xxx re=(\d+), base on python regular module, Qsqlite loop nesting function, can achieve complex web data crawling and content extraction, and then write to the database.

All new function please check the Readme file

Qsqlite is one py file only, you can just copy the Qsqlite.py file on Github to Pythonista, and then run it on python3 mode.

What

• AntPool provides a server and resource client that makes it easy and fast to set up a multi-computer distributed environment.
• AntPool provides a development library consistent with Python's native concurrent.futures.Executor asynchronous library, allowing you to quickly deploy your code to run in the multi-computer distributed environment.

Why

• In some application scenarios, we want programs to run on multiple machines in parallel to gain performance, network access improvements;
  • Computational Performance Boost: For complex computation and analysis in Python, multi-threaded Python can't boost on multi-core CPUs(limit by GIL), so Python introduced the concurrent.futures.ProcessPoolExecutor, Multi-process development is easy to implement; but when higher performance is needed, a multi-computer distributed environment quickly allows you to change just a few lines of code and take performance to new heights.
  • Network Capability Improvement: Crawling data on the Internet, analyzing and extracting it, is often limited by the traffic of the crawled server. Therefore, it is necessary to deploy the program to multiple machines with different network addresses for crawling, which is tedious and error-prone to do manually, but a multi-machine distributed environment can quickly solve such problems.
• Everyone Can Using Instead of requiring complex configurations, specialized equipment, and relearning how to develop, AntPool is available to everyone.
  • Any single device can be used as a server or resource client or both. These include PC/Mac/Linux computers, and even devices like your iPhone/iPad/Android that can run Python.
  • You can use all your devices at home to help you improve your computing performance, or you can ask your colleagues at work to contribute some of their CPU cores to help you achieve complex computations.
  • All computing resources are connected to a server-driven cluster network by running a resource client, which can be joined and quit flexibly.
  • No need to learn new concepts, as you only need to understand Python's native concurrent.futures.Executor, as the AntPool classes are same.
• Mobile Device Support iPad is a great tool, and there are already many great apps that run Python and iPython Notebook, but due to iOS security restrictions, many libraries can't be installed, Python also don't support multi-processing. With AntPool, you can call a multi-machine cluster directly on the iPad and run features on the cluster that are not available on the iPad or get more performance.
  • iPad Python Apps
    • Pythonista(requires stash , for pip support)
    • Pyto
    • CodeApp
  • iPad iPython Notebook/Jupyter Apps
    • Juno
    • Carnets

How

• Runtime requirements:
  • Python 3.5 or above
• Quick Start:
  • pip3 install antpool
  • for Pythonista user, please pip install tornado sorry for a bug, need Manual install tornado.
  • Standalone deployment server and resource client
    • Run in a terminal environment: antpoolsrv /v
    • Open a new terminal and run: antpoolclient ws://127.0.0.1:8086
    • If you need to access more resource clients to improve cluster performance, you can do so on other machines:
      • pip install antpool
      • antpoolclient ws://server IP:8086
    • Each antpoolclient client running on the same machine will use one CPU core, so if you want to exploit the performance of the whole machine, you can run multiple clients (e.g. for a 4-core CPU machine, you can run 4 clients)
  • Run the demo program
    • python3 demo1.py (demo1.py on github.com/wolf71/AntPool/demo)
  • Structure diagram
    

!!! Security Tip !!!

• The resource client is execution of the submitted modules, so the device running the resource client will be vulnerable to attacks by users with bad intentions, the system does not perform security checks on the executed modules, so it is recommended that the user running the resource client needs to verify that the user is trustworthy;
• For the sake of simplicity, the entire system currently does not use user/password authentication mechanisms (interfaces are reserved in the code); therefore, it is recommended that the server be deployed on the public network with security in mind;

User Manual

check it on GitHub

Add iPython Notebook demo.

You can using iPython Notebook on iPad with JUNO or Carnets.

check it on GitHub

• A command line tool to interactively manipulate sqlite or mysql databases for fast data processing, analysis, statistics, and graphical presentation;
  • General sqlite operations (just like the sqlite cmd tools); can be easily manipulated using sql statements.
  • Support sqlite and sqlite memory database (:memory:), support mysql database, copy whole mysql database to sqlite, copy tables between different sqlite databases.
  • Load csv or json data, or export select results to csv file.
  • Draw graphs with the data obtained by select statement, such as: scatter, line, histogram (distribution), violin graph (based on data distribution).
  • Provide a series of extended sqlite functions, supporting regular operations, text-based sum operations, Chinese ID recognition, and other extended functions.
• A script interpreter that can write script files to batch and automate a series of operations to achieve data processing, analysis, and report output;
  • Provides sql statement-based scripting capabilities, allowing you to automate batch operations and output the results to an html file.
  • Support extended loop/lend with nested support for loop statements, which is convenient for some operations that can't be handled by sql statements.
  • Support for formatting the output syntax of select results, to facilitate the generation of formatted reports.
  • The graphs are saved inline in the output html file, so that only a single html file can be sent to show all the graphs in their entirety.
• A Web Server and Job Server that can provide web or scheduled task processing backend services based on a database;
  • The web server can be scripted to enable data query and data insertion operations to facilitate the interaction of data statistics and analysis results in a browser-based.
  • Job server can be defined by scripts to achieve regular data cleaning, data aggregation, analysis, report generation, and output as local files or send emails to share the results.
• A python libray Can be imported using: from Qsqlite import Qexec , for use in python or jupyter/ipython ipynb notebooks using Qexec(cmds) calls, cmds can be a string with newlines,including a series of commands or a command.
• Summary: With sqlite's powerful sql syntax and high performance, Qsqlite hopes to enable you to efficiently use the power of sqlite and sql syntax to quickly organize, analyze, aggregation, and show result; and collaborate with Excel by exporting/importing csv files when needed to achieve greater efficiency.
  

Quickly start on Pythonista

• on pythonista , using stash, and then: pip3 install qsqlite , and then you can using it on stash by enter: qsqlite
• or just download Qsqlite.py to your pythonista file directory, and then run it.
• Demo on : https://github.com/wolf71/Qsqlite

Pythonista is best, But until now, it's not update. some bug not fix.

Now, I Just using Code. (very like VSCode, and support native Python, also numpy, pandas include) url: https://thebaselab.com/code/

same problem.
iPad Pro 11 with Apple Smart Keyboard Folio.
iPad OS 14.5.1

It's running on my MacBook Air M1.
It's all like on my iPad Pro. but faster.

Pythonista can run on MacBook Air M1, and it's fast then iPad Pro 11 ( maybe 120% speed up).

Just fork a bye-bye Feedly project ,and add some function. it’s can running on Pythonista. https://github.com/wolf71/bye-bye-feedly

1. download all your feedly account saved feeds to a json file.
2. run the websrv.py on pythonista .
3. it’s will open a web view to read all your feeds.

@mikael Thank you very much.