Python absolute import vs relative import

I've been here:

• http://www.python.org/dev/peps/pep-0328/
• http://docs.python.org/2/tutorial/modules.html#packages
• Python packages: relative imports
• python relative import example code does not work
• Ultimate answer to relative python imports
• Relative imports in Python
• Python: Disabling relative import

and plenty of URLs that I did not copy, some on SO, some on other sites, back when I thought I'd have the solution quickly.

The forever-recurring question is this: With Windows 7, 32-bit Python 2.7.3, how do I solve this "Attempted relative import in non-package" message? I built an exact replica of the package on pep-0328:

I did make functions named spam and eggs in their appropriate modules. Naturally, it didn't work. The answer is apparently in the 4th URL I listed, but it's all alumni to me. There was this response on one of the URLs I visited:

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

The above response looks promising, but it's all hieroglyphs to me. So my question, how do I make Python not return to me "Attempted relative import in non-package"? has an answer that involves -m, supposedly.

Can somebody please tell me why Python gives that error message, what it Means by non-package!, why and how do you define a 'package', and the precise answer put in terms easy enough for a kindergartener to understand. 

Edit: The imports were done from the console.

Answer:

Script vs. Module

Here's an explanation. The short version is that there is a big difference between directly running a Python file, and importing that file from somewhere else.  Just knowing what directory a file is in does not determine what package Python thinks it is in. That depends, additionally, on how you load the file into Python (by running or by importing).

There are two ways to load a Python file: as the top-level script, or as a module. A file is loaded as the top-level script if you execute it directly, for instance by typing python myfile.py on the command line. It is loaded as a module if you do python -m myfile, or if it is loaded when an import statement is encounted inside some other file. There can only be one top-level script at a time; the top-level script is the Python file you ran to start things off.

Naming

When a file is loaded, it is given a name (which is stored in its __name__ attribute). If it was loaded as the top-level script, its name is __main__. If it was loaded as a module, its name is the filename, preceded by the names of any packages/subpackages of which it is a part, separated by dots.

So for instance in your example:

package/
    __init__.py
    subpackage1/
        __init__.py
        moduleX.py
    moduleA.py

if you imported moduleX (note: imported, not directly executed), its name would be package.subpackage1.moduleX. If you imported moduleA, its name would be package.moduleA. However, if you directly run moduleX from the command line, its name will instead be __main__, and if you directly run moduleA from the command line, its name will be __main__. When a module is run as the top-level script, it loses its normal name and its name is instead __main__.

Accessing a module NOT through its containing package

There is an additional wrinkle: the module's name depends on whether it was imported "directly" from the directory it is in, or imported via a package. This only makes a difference if you run Python in a directory, and try to import a file in that same directory (or a subdirectory of it). For instance, if you start the Python interpreter in the directory package/subpackage1 and then do import moduleX, the name of moduleX will just be moduleX, and not package.subpackage1.moduleX. This is because Python adds the current directory to its search path on startup; if it finds the to-be-imported module in the current directory, it will not know that that directory is part of a package, and the package information will not become part of the module's name.

A special case is if you run the interpreter interactively (e.g., just type python and start entering Python code on the fly). In this case the name of that interactive session is __main__.

Now here is the crucial thing for your error message: if a module's name has no dots, it is not considered to be part of a package. It doesn't matter where the file actually is on disk. All that matters is what its name is, and its name depends on how you loaded it.

Now look at the quote you included in your question:

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

Relative imports...

Relative imports use the module's name to determine where it is in a package. When you use a relative import like from .. import foo, the dots indicate to step up some number of levels in the package hierarchy. For instance, if your current module's name is package.subpackage1.moduleX, then ..moduleA would mean package.moduleA. For a from .. import to work, the module's name must have at least as many dots as there are in the import statement.

... are only relative in a package

However, if your module's name is __main__, it is not considered to be in a package. Its name has no dots, and therefore you cannot use from .. import statements inside it. If you try to do so, you will get the "relative-import in non-package" error.

Scripts can't import relative

What you probably did is you tried to run moduleX or the like from the command line. When you did this, its name was set to __main__, which means that relative imports within it will fail, because its name does not reveal that it is in a package. Note that this will also happen if you run Python from the same directory where a module is, and then try to import that module, because, as described above, Python will find the module in the current directory "too early" without realizing it is part of a package.

Also remember that when you run the interactive interpreter, the "name" of that interactive session is always __main__. Thus you cannot do relative imports directly from an interactive session. Relative imports are only for use within module files.

Two solutions:

1. If you really do want to run moduleX directly, but you still want it to be considered part of a package, you can do python -m package.subpackage.moduleX. The -m tells Python to load it a s a module, not as the top-level script.
2. Or perhaps you don't actually want to run moduleX, you just want to run some other script, say myfile.py, that uses functions inside moduleX. If that is the case, put myfile.pysomewhere else --- not inside the package directory -- and run it. If inside myfile.py you do things like from package.moduleA import spam, it will work fine.

Notes

• For either of these solutions, the package directory (package in your example) must be accessible from the Python module search path (sys.path). If it is not, you will not be able to use anything in the package reliably at all.
• since Python 2.6, the module's "name" for package-resolution purposes is determined not just by its __name__ attributes but also by the __package__ attribute. That's why I'm avoiding using the explicit symbol __name__ to refer to the module's "name". Since Python 2.6 a module's "name" is effectively __package__ + '.' + __name__, or just __name__ if __package__ is None.)

Data visualization tools

seaborn -- https://seaborn.pydata.org/
lighting-viz --http://lightning-viz.org/

Numpy, how to divide the row by its sum efficiently

Method #1: use None (or np.newaxis) to add an extra dimension so that broadcasting will behave:

>>> e
array([[ 0.,  1.],
       [ 2.,  4.],
       [ 1.,  5.]])
>>> e/e.sum(axis=1)[:,None]
array([[ 0.        ,  1.        ],
       [ 0.33333333,  0.66666667],
       [ 0.16666667,  0.83333333]])

Method #2: go transpose-happy:

>>> (e.T/e.sum(axis=1)).T
array([[ 0.        ,  1.        ],
       [ 0.33333333,  0.66666667],
       [ 0.16666667,  0.83333333]])

(You can drop the axis= part for conciseness, if you want.)

Method #3: (promoted from Jaime's comment). (only works Numpy 1.7+)

Use the keepdims argument on sum to preserve the dimension:

>>> e/e.sum(axis=1, keepdims=True)
array([[ 0.        ,  1.        ],
       [ 0.33333333,  0.66666667],
       [ 0.16666667,  0.83333333]])

The Gap between two displays under MATE desktop UBUNTU

I found a related question on Ask Ubuntu which sort of did the trick for me. Instructions for the whole process, including creating the "gap" between monitors (works at least on Ubuntu 14.04): Find out the current total screen size (assuming there's currently no virtual gap between monitors): $ xrandr | grep Screen Screen 0: minimum 320 x 200, current2048 x 1280, maximum 32767 x 32767 Add the desired gap size in pixels to the width (first value displayed after current – in my case, 2048). I chose a gap of 136 pixels, so I'd have a new screen width of 2184. Set the new screen size: xrandr --fb 2184x1280 Find out the display name of your right monitor: $ xrandr | grep ^[^\(]*right HDMI3connected 1024x1280+1024+0 right (normal left inverted right x axis y axis) 376mm x 301mm Virtually move the right monitor further to the right to create the gap. Add your gap size to the third number displayed by the above command (highlighted in bold). In my example, this is 1024+136=1160: xrandr --output HDMI3 --pos 1160x0 Now, make Compiz handle the whole area as one single screen Start CompizConfig Settings Manager (installable with sudo apt-get install compizconfig-settings-manager) Go to "General Options" plugin (e.g. type this in search box) Select tab "Display Settings" Uncheck "Detect Outputs" If this doesn't already do the trick, right click on each line listed under "Outputs", select "Edit" and delete their content. When maximizing a window, it will now stretch both monitors. Fullscreen will also stretch both monitors. Clearly, this isn't ideal, as most of the time it is desirable that maximized windows fill a single monitor. This means, the "Detect Outputs" Compiz option will have to be checked and unchecked depending on what behavior is currently desired. A note to user with non-English locale: If you can't find the CompizConfig plugin and setting because you don't know how their names were translated, start CompizConfig like so:

Several Tricks

1. Print .JSON file nicely by using python:
  

python -m json.tool my_json.json

2. Read how many words in a line:

echo "abc,def,ghi"  | tr ',' ' ' | wc  

How can I reconcile detached HEAD with master/origin?

First, let’s clarify what HEAD is and what it means when it is detached. 

HEAD is the symbolic name for the currently checked out commit. When HEAD is not detached (the “normal”¹ situation: you have a branch checked out), HEAD actually points to a branch’s “ref” and the branch points to the commit. HEAD is thus “attached” to a branch. When you make a new commit, the branch that HEAD points to is updated to point to the new commit. HEAD follows automatically since it just points to the branch.

• git symbolic-ref HEAD yields refs/heads/master
  The branch named “master” is checked out.
• git rev-parse refs/heads/master yield 17a02998078923f2d62811326d130de991d1a95a
  That commit is the current tip or “head” of the master branch.
• git rev-parse HEAD also yields 17a02998078923f2d62811326d130de991d1a95a
  This is what it means to be a “symbolic ref”. It points to an object through some other reference.
  (Symbolic refs were originally implemented as symbolic links, but later changed to plain files with extra interpretation so that they could be used on platforms that do not have symlinks.)

We have HEAD → refs/heads/master → 17a02998078923f2d62811326d130de991d1a95a

When HEAD is detached, it points directly to a commit—instead of indirectly pointing to one through a branch. You can think of a detached HEAD as being on an unnamed branch. 

• git symbolic-ref HEAD fails with fatal: ref HEAD is not a symbolic ref
• git rev-parse HEAD yields 17a02998078923f2d62811326d130de991d1a95a
  Since it is not a symbolic ref, it must point directly to the commit itself.

We have HEAD → 17a02998078923f2d62811326d130de991d1a95a

The important thing to remember with a detached HEAD is that if the commit it points to is otherwise unreferenced (no other ref can reach it), then it will become “dangling” when you checkout some other commit. Eventually, such dangling commits will be pruned through the garbage collection process (by default, they are kept for at least 2 weeks and may be kept longer by being referenced by HEAD’s reflog).

¹ It is perfectly fine to do “normal” work with a detached HEAD, you just have to keep track of what you are doing to avoid having to fish dropped history out of the reflog.

---

The intermediate steps of an interactive rebase are done with a detached HEAD (partially to avoid polluting the active branch’s reflog). If you finish the full rebase operation, it will update your original branch with the cumulative result of the rebase operation and reattach HEAD to the original branch. My guess is that you never fully completed the rebase process; this will leave you with a detached HEAD pointing to the commit that was most recently processed by the rebase operation.

To recover from your situation, you should create a branch that points to the commit currently pointed to by your detached HEAD:

git branch temp
git checkout temp

_{(these two commands can be abbreviated as git checkout -b temp)}

This will reattach your HEAD to the new temp branch.

Next, you should compare the current commit (and its history) with the normal branch on which you expected to be working:

git log --graph --decorate --pretty=oneline --abbrev-commit master origin/master temp
git diff master temp
git diff origin/master temp

(You will probably want to experiment with the log options: add -p, leave off --pretty=… to see the whole log message, etc.)

If your new temp branch looks good, you may want to update (e.g.) master to point to it:

git branch -f master temp
git checkout master

_{(these two commands can be abbreviated as git checkout -B master temp)}

You can then delete the temporary branch:

git branch -d temp

Finally, you will probably want to push the reestablished history:

git push origin master

You may need to use --force to push if the remote branch can not be “fast-forwarded” to the new commit (i.e. you dropped, or rewrote some existing commit, or otherwise rewrote some bit of history).

If you were in the middle of a rebase operation you should probably clean it up. You can check whether a rebase was in process by looking for the directory .git/rebase-merge/. You can manually clean up the in-progress rebase by just deleting that directory (e.g. if you no longer remember the purpose and context of the active rebase operation). Usually you would use git rebase --abort, but that does some extra resetting that you probably want to avoid (it moves HEAD back to the original branch and resets it back to the original commit, which will undo some of the work we did above).