User Tools

Site Tools


python:python

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

python:python [2020/05/02 17:06]
python:python [2021/05/21 00:04] (current)
Line 1: Line 1:
 +====== Python ======
  
 +Man, there's too much to do and note.  Logging some stuff to investigate later...
 +
 +  * [[http://pycon.blogspot.com/2011/12/announcing-pycon-2012-tutorials.html|2011 PyCon tutorials announcement]] further details at the [[https://us.pycon.org/2012/schedule/lists/tutorials/|tutorial list]].
 +  * They mention using [[http://ipython.org/|IPython]] enhanced shell.  Hmm.
 +  * [[http://pytools.codeplex.com/releases/view/76089|Python Tools for Visual Studio 2010]].
 +  * [[http://wiki.python.org/moin/PythonSpeed/PerformanceTips|Python Performance Tips at the Python wiki]].
 +  * Of course, there's [[:xampp|getting Python running under Xampp]].
 +  * There's [[http://www.infoworld.com/d/application-development/pillars-python-six-python-web-frameworks-compared-169442|A comparison of CubicWeb, Django, Pyramid, Web.py, Web2py, and Zope 2]]
 +  * Look into the microframework [[http://flask.pocoo.org/|Flask]]. Here's a [[http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world|tutorial on making a blog with it]].
 +  * [[https://dataset.readthedocs.org/en/latest/|dataset]], an abstraction layer above relational databases.
 +  * [[http://docopt.org/|docopt]] for a stupidly easy and correct command-line interface description language.
 +  * [[http://stevenloria.com/python-best-practice-patterns-by-vladimir-keleshev-notes/|Best practices]] by Vladimir Keleshev.
 +  * [[https://twitter.com/brandon_rhodes/status/449938490516860928|You can ask a new-style class for its .__subclasses__()]] It uses weak references. Look at [[https://gist.github.com/etrepum/9862280|this interesting example with a child class inside a method]].
 +  * A Power Point deck by Alex Martelli describing [[http://www.aleax.it/yt_pydi.pdf|Dependency Injection vs. Template Method (override base method) vs. Monkey Patching]].
 +  * [[http://stackoverflow.com/questions/101268/hidden-features-of-python|Hidden Features of Python]] at StackOverflow
 +  * [[http://chriskiehl.com/article/parallelism-in-one-line|Parallelism in one line]]
 +
 +Would be nice to create a binary search in text files in Python.  Maybe based on an answer deep in [[http://stackoverflow.com/questions/744256/reading-huge-file-in-python|Reading Huge File in Python]].
 +
 +This [[http://stackoverflow.com/questions/231767/the-python-yield-keyword-explained/231855#231855|Explanation of Python "Yield"]] also mentions (at the bottom) explanations for decorators and metaclasses.
 +
 +Would be good to experiment with [[http://pingswept.org/2009/01/24/least-squares-polynomial-fitting-in-python/|least-squares polynomial fitting in Python]].
 +===== Data Analysis =====
 +
 +Looks like I should dive into:
 +
 +  * [[http://docs.scipy.org/doc/|NumPy]]
 +  * [[http://pandas.pydata.org/|Pandas]]
 +  * [[http://ipython.org/|IPython]].  (As seen above.)
 +
 +===== Template Files to Start With =====
 +
 +You have some template files in svn:
 +
 +<code>
 +/Code/python_templates/trunk/make_standalone_application.bat
 +/Code/python_templates/trunk/setup.py
 +/Code/python_templates/trunk/template_cron_job.py
 +/Code/python_templates/trunk/template_gui_app.py
 +/Code/python_templates/trunk/template_gui_app_with_worker_thread.py
 +</code>
 +
 +
 +===== For / Else (Nobreak) =====
 +
 +Python has a [[http://nedbatchelder.com/blog/201110/forelse.html|For/Else keyword]] that should have been called, "nobreak."
 +
 +===== Sorting =====
 +
 +From an old [[http://dblume.livejournal.com/28319.html|note-to-self]]...
 +<code python>
 +import operator
 +
 +rows.sort(key=operator.itemgetter(4))
 +# or
 +rows.sort(lambda x, y : x[4] == y[4] and cmp(x[2],y[2]) or cmp(x[4], y[4]))
 +</code>
 +
 +...not like I could just find the same info at the [[http://wiki.python.org/moin/HowTo/Sorting|Python wiki]] or anything. :-P
 +
 +===== Prepopulating lists with objects =====
 +
 +Remember when you lost a couple of hours thinking that the following line created a list of objects.
 +
 +<code python>
 +    l = [Obj()] * n
 +</code>
 +
 +It doesn't. It creates a list of references to one object.
 +
 +What you meant to write was this:
 +
 +<code python>
 +    l = [Obj() for _ in range(n)]
 +</code>
 +
 +===== Linux script that takes either stdin or files =====
 +
 +<code python>
 +if __name__=='__main__':
 +    if len(sys.argv) < 2:
 +        # Process lines coming from stdin.
 +        while 1:
 +            line = sys.stdin.readline()
 +            if not line:
 +                break
 +            my_process_line( line.rstrip() )
 +    else:
 +        # Process lines of the files specified.
 +        for fname in sys.argv[1:]:
 +            if not os.path.exists( fname ):
 +                treat_argument_as_literal( fname )
 +                continue
 +            with open( fname, 'r' ) as f:
 +                while 1:
 +                    line = f.readline()
 +                    if not line:
 +                        break
 +                    my_process_line( line.rstrip() )
 +</code>
 +
 +===== The With statement =====
 +
 +[[http://amix.dk/blog/post/19663#Making-ugly-code-more-beautiful-using-Pythons-with-statement|Making code more beautiful with "with"]].  (Also mentions yield.)
 +
 +===== cProfile vs. line_profiler and kernprof =====
 +
 +Raymond Hettinger [[https://twitter.com/raymondh/status/341010756101341187|likes Robert Kern's line_profiler much better than profile and cProfile]].
 +
 +Here it is: [[http://pythonhosted.org/line_profiler/|line_profiler and kernprof]]
 +
 +===== timeit =====
 +
 +<code python>
 +import timeit
 +
 +def Use_a():
 +    pass
 +
 +def Use_b():
 +    pass
 +
 +def Run_all_tests():
 +    my_setup()
 +    Use_a()
 +    Use_b()
 +
 +if __name__ == '__main__':
 +
 +#    t = timeit.timeit( 'Run_all_tests()', 'from __main__ import Run_all_tests', number=1 )
 +#    print dir( t )
 +#    print t
 +
 +    t = timeit.Timer( 'Run_all_tests()', 'from __main__ import Run_all_tests' )
 +    print t.timeit()
 +</code>
 +
 +When diving in, cProfile may come in handy.
 +
 +<code python>
 +import cProfile
 +
 +def my_function():
 +    # Complicated stuff
 +    pass
 +    
 +if __name__ == '__main__':
 +    cProfile.run( "my_function()" )
 +</code>
 +
 +===== Dynamically Calculating Column Size ======
 +
 +[[http://stackoverflow.com/questions/3685195/line-up-columns-of-numbers-print-output-in-table-format|Line up columns]]
 +
 +<code python>
 +data = '''\
 +234 127 34 23 45567
 +23 12 4 4 45
 +23456 2 1 444 567'''
 +
 +# Split input data by row and then on spaces
 +rows = [ line.strip().split(' ') for line in data.split('\n') ]
 +
 +# Reorganize data by columns
 +cols = zip(*rows)
 +
 +# Compute column widths by taking maximum length of values per column
 +col_widths = [ max(len(value) for value in col) for col in cols ]
 +
 +# Create a suitable format string
 +format = ' '.join(['%%%ds' % width for width in col_widths ])
 +
 +# Print each row using the computed format
 +for row in rows:
 +  print format % tuple(row)
 +</code>
 +
 +Which outputs:
 +<code>
 +  234 127 34  23 45567
 +   23  12  4      45
 +23456    1 444   567
 +</code>
 +
 +Also, here's a [[http://knowledgestockpile.blogspot.com/2011/01/string-formatting-in-python_09.html|nice summary of string formatting in Python]].
 +
 +===== Different Types of Objects =====
 +
 +<code python>
 +class A:
 +    """ Old, obsolete. """
 +    def __init__(self):
 +        self.__m_x = 0
 +
 +    def getx(self):
 +        return self.__m_x
 +
 +    def setx(self, x):
 +        if x < 0: x = 0
 +        self.__m_x = x
 +
 +    x = property( getx, setx )
 +
 +class B:
 +    """ Old; very small, was for for multitudes of objects. """
 +    __slots__ = [ "__m_x" ]
 +    def __init__(self):
 +        self.__m_x = 0
 +
 +    def getx(self):
 +        return self.__m_x
 +
 +    def setx(self, x):
 +        if x < 0: x = 0
 +        self.__m_x = x
 +
 +    x = property( getx, setx )        
 +
 +class C(object):
 +    """ New, reccommended. """
 +    def __init__(self):
 +        self.__x = 0
 +
 +    def getx(self):
 +        return self.__x
 +
 +    def setx(self, x):
 +        if x < 0: x = 0
 +        self.__x = x
 +
 +    x = property( getx, setx )
 +</code>
 +
 +====== filelock ======
 +
 +Evan Fosmark has a filelock module.  But here's a quick and dirty implementation of a lock that uses the current file:
 +
 +<code python>
 +import os
 +import fcntl
 +import inspect
 +# Maybe use os.path.abspath(__file__) ?
 +with open(os.path.abspath inspect.getfile(inspect.currentframe())), 'r') as f:
 +    try:
 +        fcntl.flock(f, fcntl.LOCK_EX)
 +        call_that_cannot_be_concurrent()
 +    finally:
 +        fcntl.flock(f, fcntl.LOCK_UN)
 +</code>
 +
 +====== Various Approaches to threaded URL Requests ======
 +
 +  * [[https://stackoverflow.com/questions/2632520/what-is-the-fastest-way-to-send-100-000-http-requests-in-python|Use Queue and threading's Thread]]
 +  * [[https://www.shanelynn.ie/using-python-threading-for-multiple-results-queue/|Use threading and store results in a pre-allocated list]]. Then use Queue for lots of URLs.
 +  * [[https://dev.to/rhymes/how-to-make-python-code-concurrent-with-3-lines-of-code-2fpe|Use concurrent.futures' ThreadPoolExecutor and map()]].
 +  * Or, use the doc's [[https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example|ThreadPoolExecutor Example]].
 +  * And, as mentioned in Parallelism in One Line, [[https://chriskiehl.com/article/parallelism-in-one-line|multiprocessing.dummy's Pool and its own map()]].
 +
 +====== Fibonacci Generator with Itertools ======
 +
 +<code python>
 +import itertools
 +
 +def fib(n):
 +    """Print a Fibonacci series up to n."""
 +    a, b = 0, 1
 +    while True:
 +        yield a
 +        b = a + b
 +        yield b
 +        a = a + b
 +
 +if __name__ == '__main__': 
 +    for x in itertools.islice(fib(), 5):
 +        print x
 +
 +#    for i in range( 5 ):
 +#        print i, fib( i )
 +</code>
 +
 +====== TODO ======
 +
 +TODO Link to my tips from LiveJournal and GMail, and why I chose which timing modules.