Differences

This shows you the differences between two versions of the page.

Link to this comparison view

--- python:python [2013/04/15 09:28]
dblume [Linux or Bash Tips]
+++ python:python [2020/05/02 17:06] (current)
dblume [Various Approaches to threaded URL Requests]
@@ Line 7: / Line 7: @@
   * [[http://pytools.codeplex.com/releases/view/76089|Python Tools for Visual Studio 2010]].
   * [[http://wiki.python.org/moin/PythonSpeed/PerformanceTips|Python Performance Tips at the Python wiki]].
-  * Of course, there's getting Python running under Xampp. [[xampp]]
+  * Of course, there's [[:xampp|getting Python running under Xampp]].
   * There's [[http://www.infoworld.com/d/application-development/pillars-python-six-python-web-frameworks-compared-169442|A comparison of CubicWeb, Django, Pyramid, Web.py, Web2py, and Zope 2]]
   * Look into the microframework [[http://flask.pocoo.org/|Flask]]. Here's a [[http://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world|tutorial on making a blog with it]].
+  * [[https://dataset.readthedocs.org/en/latest/|dataset]], an abstraction layer above relational databases.
+  * [[http://docopt.org/|docopt]] for a stupidly easy and correct command-line interface description language.
+  * [[http://stevenloria.com/python-best-practice-patterns-by-vladimir-keleshev-notes/|Best practices]] by Vladimir Keleshev.
+  * [[https://twitter.com/brandon_rhodes/status/449938490516860928|You can ask a new-style class for its .__subclasses__()]] It uses weak references. Look at [[https://gist.github.com/etrepum/9862280|this interesting example with a child class inside a method]].
+  * A Power Point deck by Alex Martelli describing [[http://www.aleax.it/yt_pydi.pdf|Dependency Injection vs. Template Method (override base method) vs. Monkey Patching]].
+  * [[http://stackoverflow.com/questions/101268/hidden-features-of-python|Hidden Features of Python]] at StackOverflow
+  * [[http://chriskiehl.com/article/parallelism-in-one-line|Parallelism in one line]]
 Would be nice to create a binary search in text files in Python.  Maybe based on an answer deep in [[http://stackoverflow.com/questions/744256/reading-huge-file-in-python|Reading Huge File in Python]].
@@ Line 23: / Line 30: @@
   * [[http://pandas.pydata.org/|Pandas]]
   * [[http://ipython.org/|IPython]].  (As seen above.)
 ===== Template Files to Start With =====
@@ Line 35: / Line 43: @@
 </code>
-===== Linux or Bash Tips =====
-Useful bash command for finding strings within python files...
-<code bash>
-find . -name \*.py -type f -print0 | xargs -0 grep -nI "timeit"
-</code>
-Interesting way to use ''grep -v'' to remove paths from a list generated by ''find''.  Not sure about the escaped ''|'' character, though...
-<code bash>
-#!/bin/bash
-find $PWD -regex ".*\.[hcHC]\(pp\|xx\)?" | \
-    grep -v " \|unwantedpath/unwantedpath2\|unwantedpath3" > cscope.files
-cscope -q -b
-</code>
-And these two have nothing to do with Python.  Here's how to find if a symbol is in a library, and how to search lots of object files and print the filename above the search...
-<code bash>
-nm obj-directory/libmyobject.a | c++filt | grep Initialize_my_obj
-find bindirectory/ -name \*.a -exec nm /dev/null {} \; 2>/dev/null | \
-    c++filt | grep -P "(^bindirectory.*\.a|T Initialize_my_obj)"
-</code>
-Also handy to merge two streams together...
-<code bash>
-( cat file1 && cat file2 ) | sort
-</code>
-When a little quick math is needed, use ''bc''
-<code bash>
-$ bc <<< "obase=16;ibase=10;15"
-F
-$ bc -l <<< 1/3
-.33333333333333333333
-$ bc <<< "scale=2; 1/3"
-.33
-$ bc <<< "obase=10;ibase=16;B"
-</code>
-and, when coverting from hex to dec...
-<code bash>
-echo $((0x2dec))
-</code>
-But, then again, does that really seem easier than,
-<code>
-python -c "print int('B',16)"
-</code>
-There's a bash way to calculate how many days ago a date was:
-<code bash>
-$ echo $(( ($(date +%s) - $(date -d "2012-4-16" +%s)) / 86400 ))
-</code>
-And a Python way...
-<code python>
-python -c "import datetime; print (datetime.date.today() - datetime.date( 2012, 4, 16 )).days"
-</code>
-And for displaying lines to get cut instead of wrapped:
-<code bash>
-cat_one_line_per_row() {
-  cat "$@" | expand | cut -b1-$COLUMNS
-}
-</code>
-ctags's man page says that one of its bugs is that it has too many options.  Ain't that the truth.  Make note of the obscure flag here, ''--c++-kinds=+p'', that tells ctags to process prototypes and method declarations.
-<code bash>
-ctags -n --if0=yes --c++-kinds=+p --langmap=c++:+.inl.lst \
-    --langmap=asm:+.inc --file-tags=yes -R --extra=fq \
-    --exclude=unwanted_file.lst \
-    --exclude='*unwanted-directory*/*' \
-    --regex-C++='/^.*CINIT.(.+),.*,.*,.*/CURLOPT_\1/'
-</code>
-When it's desirable to clip output to exactly the width of the window:
-<code bash>
-alias clip="expand | cut -b1-\$COLUMNS"
-</code>
 ===== For / Else (Nobreak) =====
@@ Line 186: / Line 105: @@
 [[http://amix.dk/blog/post/19663#Making-ugly-code-more-beautiful-using-Pythons-with-statement|Making code more beautiful with "with"]].  (Also mentions yield.)
+===== cProfile vs. line_profiler and kernprof =====
+Raymond Hettinger [[https://twitter.com/raymondh/status/341010756101341187|likes Robert Kern's line_profiler much better than profile and cProfile]].
+Here it is: [[http://pythonhosted.org/line_profiler/|line_profiler and kernprof]]
 ===== timeit =====
@@ Line 307: / Line 232: @@
     x = property( getx, setx )
+</code>
+====== filelock ======
+Evan Fosmark has a filelock module.  But here's a quick and dirty implementation of a lock that uses the current file:
+<code python>
+import os
+import fcntl
+import inspect
+# Maybe use os.path.abspath(__file__) ?
+with open(os.path.abspath inspect.getfile(inspect.currentframe())), 'r') as f:
+    try:
+        fcntl.flock(f, fcntl.LOCK_EX)
+        call_that_cannot_be_concurrent()
+    finally:
+        fcntl.flock(f, fcntl.LOCK_UN)
+</code>
+====== Various Approaches to threaded URL Requests ======
+  * [[https://stackoverflow.com/questions/2632520/what-is-the-fastest-way-to-send-100-000-http-requests-in-python|Use Queue and threading's Thread]]
+  * [[https://www.shanelynn.ie/using-python-threading-for-multiple-results-queue/|Use threading and store results in a pre-allocated list]]. Then use Queue for lots of URLs.
+  * [[https://dev.to/rhymes/how-to-make-python-code-concurrent-with-3-lines-of-code-2fpe|Use concurrent.futures' ThreadPoolExecutor and map()]].
+  * Or, use the doc's [[https://docs.python.org/3/library/concurrent.futures.html#threadpoolexecutor-example|ThreadPoolExecutor Example]].
+  * And, as mentioned in Parallelism in One Line, [[https://chriskiehl.com/article/parallelism-in-one-line|multiprocessing.dummy's Pool and its own map()]].
+====== Fibonacci Generator with Itertools ======
+<code python>
+import itertools
+def fib(n):
+    """Print a Fibonacci series up to n."""
+    a, b = 0, 1
+    while True:
+        yield a
+        b = a + b
+        yield b
+        a = a + b
+if __name__ == '__main__':
+    for x in itertools.islice(fib(), 5):
+        print x
+#    for i in range( 5 ):
+#        print i, fib( i )
 </code>