• Matteo

@JonB ok with scripts, but I'm searching for a way to use only Pythonista console with sage sarver (it is sometimes more convenient and fast to stay in console environment than to swipe from/to console and script editor).

Sorry for patience, I ask if Pythonista console module (ore some extra objc code) can be used by a wrench action to read console input written by user without pressing the return key on keyboard. I mean:

1. I write in console the following:
>>> a=1.5

1. I don't press return key on keyboard but I press a wrench action C that reads the input and save it as a string in a variable (for example named "console_input"), so regardless of whether or not I pressed the return button on keyboard, if I write in console:
>>> console_input


and press return, I'd like to see the output as a string (not evaluated, like if I do print('a=1.5')):

>>> a=1.5


Can it be possible?

Thank you
Regards

• Matteo

@JonB and @dgelessus, thank you for reply, yes my ultimate target is to write something (I don't know if I will have something working) for the Pythonista console able to send strings to sagemathcell server instead of execute them with internal python interpreter and to post-process the output in order to print the output from sage as string in the console (as described in my first example above).

Example of possible usage:

1. user enables the by-pass of internal interpreter with the execution of a specific script A as a wrench script
2. in this way the return key, when touched on the keyboard, doesn't call the internal python interpreter but reads the full string (the command written by user) and sends it to sage with all required arguments, waits for output and prints any output (output, warnings, errors, etc...) as string with images if existing.
3. user can decide if evaluate the output in order to convert it as a number, array, etc...
4. when user wants to return to use internal interpreter, a second script B should be executed always through the wrench menu (or the @JonB's StatusBarOverlay.py).

Like the following command in Pythonista console (as example):

>>> minimize(fun,x0,...)  ## from scipy.optimize


and the touch of the return key should send the string "minimize(fun,x0,...)" to server with all explicit arguments (fun, X0, args, method, etc...) after entering in the cell string the string "from scipy.optimize import minimize" for proper importing of the required module.

I asked about the existence of some objc commands to by-pass internal interpreter when user touches the return key after inserting the command in the console. Other option is to be able to install in Pythonista some specific library like fire, for example (but it seems to me not pure python, even if GitHub says it is 100% python: I tried to install it in Pythonista but it doesn't work).

Other option is to program a user key with @JonB StatusBarOverlay.py, it is what I will do. So (I hope) I will have a user defined key in upper status bar that simulates the return key of the built-in keyboard but with a specific pre-processing and post-processing action.

But I still have the curiosity to know if with Pythonista (and its several surprises) user can customize the effect of the return key touching in console environment.

Thank you for numerous tips and explanations.
Regards

• Matteo

Hi, do you know a way (with some esoteric objc maybe?) to bypass (prevent the execution of) the Pythonista internal python interpreter when user presses the key return in built-in keyboard in the Python console after writing a command, in order to run other external scripts written by user?

For example: when I'm in the Pythonista console and I write:

>>> print('hello')  ## and press "return" key in the keyboard


I'd like to force Pythonista to execute a script X instead of the normal execution (that is the execution of the command "print('hello')" by the internal python interpreter of Pythonista). So, following this example, I don't want the common output that is:

>>> hello


but I'd like to have, as output and as a simple example, the number of single characters in the string of the command "print('hello')" (if the script X is a script that reads the string written by user and counts its characters). So when I press return key, I'd like to see in console the output:

>>> 14


that are the number of characters of the input string "print('hello')".

The script X that counts the characters of the user input string in console is only an example, only to explain better my question.

Thank you
Regards

• Matteo

@lpl Hi, in my opinion (my opinion tries always to be in favor of democracy) it is better if Pythonista is not so powerful, because otherwise there would be very little chances for other developers. In this way other people can develop apps that try to be closer to users needs (let's say that this creates a good and lively market).

The main vocation of Pythonista is not to be a scientific tool (we all understood it), but an advanced IDE with a lot of features.

omz, the developer is very perfectionist in the things he does (it is my idea) and I suppose he uses a lot of time to plan very well all his development ideas about his app. In other words, I think that to add a c based python library to Pythonista is not so difficult for him, but it is not the main target in his idea about Pythonista and, also, adding c libraries in Pythonista would inevitably give the developer extra responsibility for keeping these libraries up-to-date (it is not to be taken for granted that the updated version of a c-based python library can be easily compiled like the previous one).

The best thing for now could be to install different python IDEs available on Apple Store (everyone has to decide his/her budget: 3$, 15$, 300$, ...) that perform different and specific tasks, and even better if all these apps/IDEs can access to user's iCloud folder, in your device you could run what you want with the IDE that best meets your needs. If you don't want to install other python apps/IDEs, try to develop something about the sage interface in order to allow user to use pandas in Pythonista console using the remote server but in a way that is as if pandas were really installed in Pythonista, I mean, in order to be able to execute in console each pandas command with any argument and each single pandas command returns the output in console. I would be grateful to you, because I do not know where to start doing something like that. Example of session in Pythonista console with pandas available in sage remote server: >>> data = pandas.read_csv("mydata.csv") ## here Pythonista sends the pandas command to server with the argument (automatically, the argument 'mydata.csv' is obviously in Pythonista folder, in this case it is in main Documents folder, but you could add relative path for other files) and wait for answer in order to save in data the output >>> data.head() ## here Pythonista print data content  posted in Pythonista read more • Matteo @wpeter11 Hi, I'm happy to know it helps you, if you want it would be a good thing to write a "thank you" mail to Andrey, that works continuously on SageMathCell (William A. Stein is the founder of SageMath). A lot of people work on it. Obviously it is not so convenient to compile (yes sagemathcell allows you to compile and run C and Fortran codes and to use common Linux shell commands like ls with !ls, for example) every time a python lib, but you can do it the first time, create a zip folder with all compiled (for Linux) libraries, download the zip to upload it in your preferred cloud service (dropbox, github, ....), and create a python script that, when you need it, allows you to download in a math session of sagemathcell your zipped compiled library (with dependencies), extract it and use it. If you have limit about length of your code, you can save it in a file, copy the file in cloud service, download it when you need and open it to read content in sage server (without sending the full code to server by inserting the text in the cell). I don't know if with python script sage_interface user can do several calculations maintaining the same temporary/sandboxed folder created by sagemathcell. I think it is possible but some tests are required. For now every time you run a script sent to the server, sage_interface creates each time a new cell (that is: a new different temp folder in the server machines, and that folder is visible only by you, in theory). This means that you should download or compile every time what you want, also for simple tests. @JonB Hi, yes it is a nice thing, I discovered it a short time ago when I was doing tests on file creation in sagemathcell temp folders. It is possibile to use pip and it seems to call automatically gcc/fortran compiler if needed (gcc available as a built-in package). Only for Linux. It is like to have in Pythonista a free Linux virtual machine available online. Regards posted in Pythonista read more • Matteo @wpeter11 Hi, if you want to test some scripts that require astropy (or other not pure modules, but you should try to install them because I'm not sure the following procedure works for all not-pure python libraries) you can use sagemathcell, as @JonB said (but you don't need your own sagemath server on your pc), with Pythonista (sagemathcell is an online python interpreter with strong features...it is free and maintained by Andrey Novoseltsev and some other people, with machines and electricity power from some places around the world). Remember sagemathcell runs python 2.7.xx (I can't remember, but you can query the version). Try to Execute the following code in the webbrowser, or save it as a script in Pythonista by following procedure here or by using JonB wrench version of sage_interface. ## first you need to install the lib because it is not a built-in lib in sagemathcell !pip install "--target=astropy" astropy sys.path.append(os.getcwd() + "/astropy") ## then you can test any example: you will have an error after some calculations, I don't know why but probably it is due to python 2.7 incompatibility with last version of astropy, I'm not sure, you should do some experiments. the error is "Coordinate frame got unexpected keywords: ['pm_Beta', 'pm_Lambda_cosBeta']". Try to solve it by yourself, sorry. ##### Example from "http://docs.astropy.org/en/stable/generated/examples/coordinates/plot_sgr-coordinate-frame.html#sphx-glr-generated-examples-coordinates-plot-sgr-coordinate-frame-py" # -*- coding: utf-8 -*- """ ========================================================== Create a new coordinate class (for the Sagittarius stream) ========================================================== This document describes in detail how to subclass and define a custom spherical coordinate frame, as discussed in :ref:astropy-coordinates-design and the docstring for ~astropy.coordinates.BaseCoordinateFrame. In this example, we will define a coordinate system defined by the plane of orbit of the Sagittarius Dwarf Galaxy (hereafter Sgr; as defined in Majewski et al. 2003). The Sgr coordinate system is often referred to in terms of two angular coordinates, :math:\Lambda,B. To do this, wee need to define a subclass of ~astropy.coordinates.BaseCoordinateFrame that knows the names and units of the coordinate system angles in each of the supported representations. In this case we support ~astropy.coordinates.SphericalRepresentation with "Lambda" and "Beta". Then we have to define the transformation from this coordinate system to some other built-in system. Here we will use Galactic coordinates, represented by the ~astropy.coordinates.Galactic class. See Also -------- * The gala package <http://gala.adrian.pw/>_, which defines a number of Astropy coordinate frames for stellar stream coordinate systems. * Majewski et al. 2003, "A Two Micron All Sky Survey View of the Sagittarius Dwarf Galaxy. I. Morphology of the Sagittarius Core and Tidal Arms", https://arxiv.org/abs/astro-ph/0304198 * Law & Majewski 2010, "The Sagittarius Dwarf Galaxy: A Model for Evolution in a Triaxial Milky Way Halo", https://arxiv.org/abs/1003.1132 * David Law's Sgr info page http://www.stsci.edu/~dlaw/Sgr/ ------------------- *By: Adrian Price-Whelan, Erik Tollerud* *License: BSD* ------------------- """ ############################################################################## # Make print work the same in all versions of Python, set up numpy, # matplotlib, and use a nicer set of plot parameters: import numpy as np import matplotlib.pyplot as plt from astropy.visualization import astropy_mpl_style plt.style.use(astropy_mpl_style) ############################################################################## # Import the packages necessary for coordinates from astropy.coordinates import frame_transform_graph from astropy.coordinates.matrix_utilities import rotation_matrix, matrix_product, matrix_transpose import astropy.coordinates as coord import astropy.units as u ############################################################################## # The first step is to create a new class, which we'll call # Sagittarius and make it a subclass of # ~astropy.coordinates.BaseCoordinateFrame: class Sagittarius(coord.BaseCoordinateFrame): """ A Heliocentric spherical coordinate system defined by the orbit of the Sagittarius dwarf galaxy, as described in http://adsabs.harvard.edu/abs/2003ApJ...599.1082M and further explained in http://www.stsci.edu/~dlaw/Sgr/. Parameters ---------- representation : BaseRepresentation or None A representation object or None to have no data (or use the other keywords) Lambda : Angle, optional, must be keyword The longitude-like angle corresponding to Sagittarius' orbit. Beta : Angle, optional, must be keyword The latitude-like angle corresponding to Sagittarius' orbit. distance : Quantity, optional, must be keyword The Distance for this object along the line-of-sight. pm_Lambda_cosBeta : :class:~astropy.units.Quantity, optional, must be keyword The proper motion along the stream in Lambda (including the cos(Beta) factor) for this object (pm_Beta must also be given). pm_Beta : :class:~astropy.units.Quantity, optional, must be keyword The proper motion in Declination for this object (pm_ra_cosdec must also be given). radial_velocity : :class:~astropy.units.Quantity, optional, must be keyword The radial velocity of this object. """ default_representation = coord.SphericalRepresentation default_differential = coord.SphericalCosLatDifferential frame_specific_representation_info = { coord.SphericalRepresentation: [ coord.RepresentationMapping('lon', 'Lambda'), coord.RepresentationMapping('lat', 'Beta'), coord.RepresentationMapping('distance', 'distance')] } ############################################################################## # Breaking this down line-by-line, we define the class as a subclass of # ~astropy.coordinates.BaseCoordinateFrame. Then we include a descriptive # docstring. The final lines are class-level attributes that specify the # default representation for the data, default differential for the velocity # information, and mappings from the attribute names used by representation # objects to the names that are to be used by the Sagittarius frame. In this # case we override the names in the spherical representations but don't do # anything with other representations like cartesian or cylindrical. # # Next we have to define the transformation from this coordinate system to some # other built-in coordinate system; we will use Galactic coordinates. We can do # this by defining functions that return transformation matrices, or by simply # defining a function that accepts a coordinate and returns a new coordinate in # the new system. Because the transformation to the Sagittarius coordinate # system is just a spherical rotation from Galactic coordinates, we'll just # define a function that returns this matrix. We'll start by constructing the # transformation matrix using pre-deteremined Euler angles and the # rotation_matrix helper function: SGR_PHI = (180 + 3.75) * u.degree # Euler angles (from Law & Majewski 2010) SGR_THETA = (90 - 13.46) * u.degree SGR_PSI = (180 + 14.111534) * u.degree # Generate the rotation matrix using the x-convention (see Goldstein) D = rotation_matrix(SGR_PHI, "z") C = rotation_matrix(SGR_THETA, "x") B = rotation_matrix(SGR_PSI, "z") A = np.diag([1.,1.,-1.]) SGR_MATRIX = matrix_product(A, B, C, D) ############################################################################## # Since we already constructed the transformation (rotation) matrix above, and # the inverse of a rotation matrix is just its transpose, the required # transformation functions are very simple: @frame_transform_graph.transform(coord.StaticMatrixTransform, coord.Galactic, Sagittarius) def galactic_to_sgr(): """ Compute the transformation matrix from Galactic spherical to heliocentric Sgr coordinates. """ return SGR_MATRIX ############################################################################## # The decorator @frame_transform_graph.transform(coord.StaticMatrixTransform, # coord.Galactic, Sagittarius) registers this function on the # frame_transform_graph as a coordinate transformation. Inside the function, # we simply return the previously defined rotation matrix. # # We then register the inverse transformation by using the transpose of the # rotation matrix (which is faster to compute than the inverse): @frame_transform_graph.transform(coord.StaticMatrixTransform, Sagittarius, coord.Galactic) def sgr_to_galactic(): """ Compute the transformation matrix from heliocentric Sgr coordinates to spherical Galactic. """ return matrix_transpose(SGR_MATRIX) ############################################################################## # Now that we've registered these transformations between Sagittarius and # ~astropy.coordinates.Galactic, we can transform between *any* coordinate # system and Sagittarius (as long as the other system has a path to # transform to ~astropy.coordinates.Galactic). For example, to transform from # ICRS coordinates to Sagittarius, we would do: icrs = coord.ICRS(280.161732*u.degree, 11.91934*u.degree) sgr = icrs.transform_to(Sagittarius) print(sgr) ############################################################################## # Or, to transform from the Sagittarius frame to ICRS coordinates (in this # case, a line along the Sagittarius x-y plane): sgr = Sagittarius(Lambda=np.linspace(0, 2*np.pi, 128)*u.radian, Beta=np.zeros(128)*u.radian) icrs = sgr.transform_to(coord.ICRS) print(icrs) ############################################################################## # As an example, we'll now plot the points in both coordinate systems: fig, axes = plt.subplots(2, 1, figsize=(8, 10), subplot_kw={'projection': 'aitoff'}) axes[0].set_title("Sagittarius") axes[0].plot(sgr.Lambda.wrap_at(180*u.deg).radian, sgr.Beta.radian, linestyle='none', marker='.') axes[1].set_title("ICRS") axes[1].plot(icrs.ra.wrap_at(180*u.deg).radian, icrs.dec.radian, linestyle='none', marker='.') plt.show() ############################################################################## # This particular transformation is just a spherical rotation, which is a # special case of an Affine transformation with no vector offset. The # transformation of velocity components is therefore natively supported as # well: sgr = Sagittarius(Lambda=np.linspace(0, 2*np.pi, 128)*u.radian,Beta=np.zeros(128)*u.radian,pm_Lambda_cosBeta=np.random.uniform(-5, 5, 128)*u.mas/u.yr,pm_Beta=np.zeros(128)*u.mas/u.yr) icrs = sgr.transform_to(coord.ICRS) print(icrs) fig, axes = plt.subplots(3, 1, figsize=(8, 10), sharex=True) axes[0].set_title("Sagittarius") axes[0].plot(sgr.Lambda.degree, sgr.pm_Lambda_cosBeta.value, linestyle='none', marker='.') axes[0].set_xlabel(r"$\Lambda$[deg]") axes[0].set_ylabel(r"$\mu_\Lambda \, \cos B$[{0}]" .format(sgr.pm_Lambda_cosBeta.unit.to_string('latex_inline'))) axes[1].set_title("ICRS") axes[1].plot(icrs.ra.degree, icrs.pm_ra_cosdec.value, linestyle='none', marker='.') axes[1].set_ylabel(r"$\mu_\alpha \, \cos\delta$[{0}]" .format(icrs.pm_ra_cosdec.unit.to_string('latex_inline'))) axes[2].set_title("ICRS") axes[2].plot(icrs.ra.degree, icrs.pm_dec.value, linestyle='none', marker='.') axes[2].set_xlabel("RA [deg]") axes[2].set_ylabel(r"$\mu_\delta\$ [{0}]"
.format(icrs.pm_dec.unit.to_string('latex_inline')))

plt.show()


Bye

• Matteo

@JonB Hi, ok thank you for useful info, I’m testing pysandbox in a virtual machine on my pc.

If I will be satisfied with some tests I want to perform (for example the execution of a script that tries to read something out the folder in which it is saved but that can import any modules from external standard ‘site-packages’ folder), then I will install pysandbox also in Pythonista.

Obviously if the script I try to run can’t import external modules, it is not so easy/useful to use (in my opinion).

A solution could be to copy manually inside the folder (where there is the potentially dangerous script to be executed) all the required modules by copying them from folder site-packages: in this way, if there are all the required modules in the same folder, maybe the script can be run.

In other words, a potentially dangerous script should always, in my opinion, be able to read modules in site-packages (but not be able to modify or delete them if user wants it), but should not read or modify or delete files or modules in other folders that are set "private" by the user (I will try to understand if it is possibile with pysandbox, and I wanted also that it could be done manually in Pythonista settings, and it is the reason I started all the discussion).

I still have to understand also if pysandbox creates a sandbox environment only for the potentially dangerous scripts or for the scripts (or files or folders) that I want to protect by the potentially dangerous scripts.

These are the arguments I'm studying in these period (in my spare time) to understand how the iOS sandbox feature works (as a personal curiosity).

For now I’d like to ask you other things I’m interested about:

1. Do you know if it is possible for any app (installed from Apple Store) to read or delete files and folders in Pythonista main folder if that app knows the full and absolute path of my Pythonista ‘Documents’ folder that is something like /private/var/......../Documents/...? If yes, why it would be possible? Where is the iOS sandbox feature in this situation? I mean, if I know where to search, the sandbox feature should in my opinion give me no chance to access to sandboxed folders and files even if I know where to search, as for a thief who knows where is my house (the full absolute path) and tries to opens the door that is closed with a very strong lock (the read-write protection by sandboxing, with password request for read-write access, for example).

2. Do you know if a python script (and if yes how could this script be written) can read or delete files and folders in Pythonista ‘Documents’ main folder if that script knows its full and absolute path, like for the app of point 1? Also in this situation, how can user defend own files? Only with backup? But if the user is only afraid that the files will be read without his/her knowledge by that script? In other words, can a python script search not in subfolders, starting from a folder, but in upper/parent folders in a recursive way in order to try to read confidential files saved in folders outside the one in which this script is executed?

Sorry for these questions, but I'm mainly interested about these arguments in this period and I find very nice to discuss these things with the people of this forum, that are very skilled but also very humble in giving information.

Thank you
Regards

• Matteo

@JonB Ok understood!

About pysandbox, yes I'm agree with you that pysandbox doesn't delete anything, the problem is that if I want to test pysandbox on Pythonista2 by activate some sandbox features in order to try to create a sandbox environment in Pythonista2, I definitely would try the effectiveness of the system by running the script I wrote in post #12 and if I did something wrong with pysandbox setting I would delete everything in both Pythonista2 and Pythonista3 (yes I could perform a backup of all and move it somewhere...). So no problem with pysandbox, the problem exists if I test, by trying to delete something, pysandbox effectiveness.

About your code, to know that you knows these things surprises me every time. I will try the code in both Pythonista2 and 3 by moving some files of a test folder in private folder (following your instructions), in order to see if I can work easly with these files also using StaSh (and giving up the convenient file browser of Pythonista). If I will find it hard, I will not worry anymore and I will try some suspicious scripts with some other different python app (in app store there are some of them but you can not compare them with Pythonista) or with sagecell.

Last thing: I was interested in the question mainly from a practical point of view (I wanted to have a solution that made Pythonista3 invisible from Pythonista2); then the interest became more theoretical and I asked myself if at the programming level of the Pythonista3 application it is possible for @omz to add an option that allows the user to choose whether to have a shared folder between Pythonista2 and 3 or not, so, if not, the Pythonista3 content can not be seen in any way by Pythonista2 (even by using strange objc codes executed on Pythonista2).

Thank you again
Regards

• Matteo

@JonB Hi, first of all thank you for your continuous suggestions, much appreciated.

Unfortunately I'm so ignorant that the code you wrote for me is like a mixture of Burmese, Cyrillic and Chinese. Sorry for this

I can't understand what works about the code you wrote, that is: if I run that script in Pythonista code editor it does not work while if I run it in StaSh it works?
And what should I see? That Pythonista3 becomes invisible with respect to Pythonista2 after the execution? So, if I'm right, by executing that strange objc code in StaSh I can hide Pythonista3 from Pythonista2 and the code you wrote can be executed only in StaSh (and this prevents any other script executed in Pythonista from bypassing the changes made by running your script)?

Thank you JonB for you patience, and sorry if I have some requests that can't be converted in a simple code or setting.

Regards

PS: I will try surely pysandbox but not with Pythonista2 or 3 because I know that if I do some experiments with it (for example: file deletion test in a sandboxed folder created with pysandbox), I could delete all my Pythonista3 content also if I use Pythonista2. I will try pysandbox in a virtual machine with a standard python distribution by using my PC.

• Matteo

@JonB, Hello, thank you so much for your info.

So I learned something new, but why I can't set in Pythonista3 an option (manually and only manually) in order to hide main folder of Pythonista3 when I'm using Pythonista2?

I know I can see Pythonista2 files (and copy them) from Pythonista3 if I set the option "Pythonista 2 Files" in Pythonista3 settings, but I find it strange that I can't hide Pythonista3 from Pythonista2 if I want it.

If I have both Pythonista2 and Pythonista3 installed on my iPhone, I'm using Pythonista2 and I run the following script in Pythonista2 (with top = '/'),

import os
top = '#!#!# warning!!!'
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
os.remove(os.path.join(root, name))
for name in dirs:
os.rmdir(os.path.join(root, name))


I suppose it deletes all files also in Pythonista3, but I don't want it. If the two apps are independent from a ios point of view, it would better in my opinion to allow user to set or unset the possibility to see Pythonista3 content and modify it from Pythonista2 (only for security reasons).

I can understand it is useful to have a shared folder for both apps, but at least user should decide, with a manual option in Pythonista3 (only manual), if Pythonista3 can share the same user folder with other apps, first of all Pythonista2; this in order to have a full sandbox environment in the same idevice for those who are lucky enough to have both Pythonista2 and Pythonista3 installed.

Hi @cvp, thank you for info about filenav, it is also a powerful tool like PhoneManager, but I'm really not interested in choose a file manager for Pythonista, the built-in one is enough for me. I referred to PhoneManager program only to prove that when user run PhoneManager (and maybe filenav) in Pythonista2, the program can see content of Pythonista3, because (@JonB said) the two apps share the same user writable folder. But I don't like this choice at all, at least I'd like to be able to choose to have a shared folder between the two apps or to hide all content of Pythonista3 from full Pythonista2 app, with an option that can only be set manually.

Have you some other hints or it is impossible to hide Pythonista3 content from Pythonista2 or other apps?

Thank you
Regards