[Stackless-checkins] CVS: slpdev/src/2.2/src/Misc NEWS.help, NONE, 1.1 SpecialBuilds.txt, NONE, 1.1 pymemcompat.h, NONE, 1.1
Christian Tismer
tismer at centera.de
Sat May 1 02:54:40 CEST 2004
- Previous message: [Stackless-checkins] CVS: slpdev/src/2.2/src/Mac/scripts BuildApplet.icns, NONE, 1.1 BuildApplet.plist, NONE, 1.1 buildpkg.py, NONE, 1.1 cachersrc.py, NONE, 1.1 genallsuites.py, NONE, 1.1 mkestrres-macerrors.h, NONE, 1.1
- Next message: [Stackless-checkins] CVS: slpdev/src/2.2/src/Misc/RPM python-2.3.spec, NONE, 1.1
- Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
Update of /home/cvs/slpdev/src/2.2/src/Misc
In directory centera.de:/home/tismer/slpdev/src/2.2/src/Misc
Added Files:
NEWS.help SpecialBuilds.txt pymemcompat.h
Log Message:
added files
--- NEW FILE: NEWS.help ---
-*- text -*-
If you edited Misc/NEWS before it was converted to ReST format skimming this
file should help make the transition a bit easier. For full details about
Docutils and ReST, go to the Docutils website:
http://docutils.sourceforge.net/
To process Misc/NEWS using Docutils, you'll need the latest docutils
snapshot:
http://docutils.sf.net/docutils-snapshot.tgz
Docutils works with Python 2.1 or newer (including 2.3).
To process NEWS into NEWS.html, first install Docutils, and then run
this command:
python .../docutils/tools/html.py NEWS >NEWS.html
Here ".../docutils" is the directory into which the above snapshot was
extracted. (I hope this recipe will change for the better.)
David Goodger made a change to the allowable structure of internal
references which greatly simplified initial conversion of the file.
The changes required fell into the following categories:
* The top-level "What's New" section headers changed to:
What's New in Python 2.3 alpha 1?
=================================
*Release date: DD-MMM-2002*
Note that the release date line is emphasized, with a "*" at each
end.
* Subsections are underlined with a single row of hyphens:
Type/class unification and new-style classes
--------------------------------------------
* Places where "balanced" single quotes were used were changed to use
apostrophes as both the opening and closing quote (`string' -> 'string').
* In a few places asterisks needed to be escaped which would otherwise have
been interpreted as beginning blocks of italic or bold text, e.g.:
- The type of tp_free has been changed from "``void (*)(PyObject *)``"
to "``void (*)(void *)``".
Note that only the asterisks preceded by whitespace needed to be escaped.
* One instance of a word ending with an underscore needed to be quoted
("PyCmp_" became "``PyCmp_``").
* One table was converted to ReST form (search Misc/NEWS for "New codecs"
for this example).
* A few places where chunks of code or indented text were displayed needed
to be properly introduced (preceding paragraph terminated by "::" and the
chunk of code or text indented w.r.t. the paragraph). For example:
- Note that PyLong_AsDouble can fail! This has always been true,
but no callers checked for it. It's more likely to fail now,
because overflow errors are properly detected now. The proper way
to check::
double x = PyLong_AsDouble(some_long_object);
if (x == -1.0 && PyErr_Occurred()) {
/* The conversion failed. */
}
--- NEW FILE: SpecialBuilds.txt ---
This file describes some special Python build types enabled via
compile-time preprocessor defines.
---------------------------------------------------------------------------
Py_REF_DEBUG introduced in 1.4
named REF_DEBUG before 1.4
Turn on aggregate reference counting. This arranges that extern
_Py_RefTotal hold a count of all references, the sum of ob_refcnt across
all objects. In a debug-mode build, this is where the "8288" comes from
in
>>> 23
23
[8288 refs]
>>>
Note that if this count increases when you're not storing away new objects,
there's probably a leak. Remember, though, that in interactive mode the
special name "_" holds a reference to the last result displayed!
Py_REF_DEBUG also checks after every decref to verify that the refcount
hasn't gone negative, and causes an immediate fatal error if it has.
Special gimmicks:
sys.gettotalrefcount()
Return current total of all refcounts.
Available under Py_REF_DEBUG in Python 2.3.
Before 2.3, Py_TRACE_REFS was required to enable this function.
---------------------------------------------------------------------------
Py_TRACE_REFS introduced in 1.4
named TRACE_REFS before 1.4
Turn on heavy reference debugging. This is major surgery. Every PyObject
grows two more pointers, to maintain a doubly-linked list of all live
heap-allocated objects. Most builtin type objects are not in this list,
as they're statically allocated. Starting in Python 2.3, if COUNT_ALLOCS
(see below) is also defined, a static type object T does appear in this
list if at least one object of type T has been created.
Note that because the fundamental PyObject layout changes, Python modules
compiled with Py_TRACE_REFS are incompatible with modules compiled without
it.
Py_TRACE_REFS implies Py_REF_DEBUG.
Special gimmicks:
sys.getobjects(max[, type])
Return list of the (no more than) max most-recently allocated objects,
most recently allocated first in the list, least-recently allocated
last in the list. max=0 means no limit on list length.
If an optional type object is passed, the list is also restricted to
objects of that type.
The return list itself, and some temp objects created just to call
sys.getobjects(), are excluded from the return list. Note that the
list returned is just another object, though, so may appear in the
return list the next time you call getobjects(); note that every
object in the list is kept alive too, simply by virtue of being in
the list.
envar PYTHONDUMPREFS
If this envar exists, Py_Finalize() arranges to print a list of
all still-live heap objects. This is printed twice, in different
formats, before and after Py_Finalize has cleaned up everything it
can clean up. The first output block produces the repr() of each
object so is more informative; however, a lot of stuff destined to
die is still alive then. The second output block is much harder
to work with (repr() can't be invoked anymore -- the interpreter
has been torn down too far), but doesn't list any objects that will
die. The tool script combinerefs.py can be run over this to combine
the info from both output blocks. The second output block, and
combinerefs.py, were new in Python 2.3b1.
---------------------------------------------------------------------------
PYMALLOC_DEBUG introduced in 2.3
When pymalloc is enabled (WITH_PYMALLOC is defined), calls to the PyObject_
memory routines are handled by Python's own small-object allocator, while
calls to the PyMem_ memory routines are directed to the system malloc/
realloc/free. If PYMALLOC_DEBUG is also defined, calls to both PyObject_
and PyMem_ memory routines are directed to a special debugging mode of
Python's small-object allocator.
This mode fills dynamically allocated memory blocks with special,
recognizable bit patterns, and adds debugging info on each end of
dynamically allocated memory blocks. The special bit patterns are:
#define CLEANBYTE 0xCB /* clean (newly allocated) memory */
#define DEADBYTE 0xDB /* dead (newly freed) memory */
#define FORBIDDENBYTE 0xFB /* fordidden -- untouchable bytes */
Strings of these bytes are unlikely to be valid addresses, floats, or 7-bit
ASCII strings.
8 bytes are added at each end of each block of N bytes requested. The
memory layout is like so, where p represents the address returned by a
malloc-like or realloc-like function (p[i:j] means the slice of bytes
from *(p+i) inclusive up to *(p+j) exclusive; note that the treatment
of negative indices differs from a Python slice):
p[-8:-4]
Number of bytes originally asked for. 4-byte unsigned integer,
big-endian (easier to read in a memory dump).
p[-4:0]
Copies of FORBIDDENBYTE. Used to catch under- writes and reads.
p[0:N]
The requested memory, filled with copies of CLEANBYTE, used to catch
reference to uninitialized memory.
When a realloc-like function is called requesting a larger memory
block, the new excess bytes are also filled with CLEANBYTE.
When a free-like function is called, these are overwritten with
DEADBYTE, to catch reference to freed memory. When a realloc-
like function is called requesting a smaller memory block, the excess
old bytes are also filled with DEADBYTE.
p[N:N+4]
Copies of FORBIDDENBYTE. Used to catch over- writes and reads.
p[N+4:N+8]
A serial number, incremented by 1 on each call to a malloc-like or
realloc-like function.
4-byte unsigned integer, big-endian.
If "bad memory" is detected later, the serial number gives an
excellent way to set a breakpoint on the next run, to capture the
instant at which this block was passed out. The static function
bumpserialno() in obmalloc.c is the only place the serial number
is incremented, and exists so you can set such a breakpoint easily.
A realloc-like or free-like function first checks that the FORBIDDENBYTEs
at each end are intact. If they've been altered, diagnostic output is
written to stderr, and the program is aborted via Py_FatalError(). The
other main failure mode is provoking a memory error when a program
reads up one of the special bit patterns and tries to use it as an address.
If you get in a debugger then and look at the object, you're likely
to see that it's entirely filled with 0xDB (meaning freed memory is
getting used) or 0xCB (meaning uninitialized memory is getting used).
Note that PYMALLOC_DEBUG requires WITH_PYMALLOC.
Special gimmicks:
envar PYTHONMALLOCSTATS
If this envar exists, a report of pymalloc summary statistics is
printed to stderr whenever a new arena is allocated, and also
by Py_Finalize().
---------------------------------------------------------------------------
Py_DEBUG introduced in 1.5
named DEBUG before 1.5
This is what is generally meant by "a debug build" of Python.
Py_DEBUG implies LLTRACE, Py_REF_DEBUG, Py_TRACE_REFS, and
PYMALLOC_DEBUG (if WITH_PYMALLOC is enabled). In addition, C
assert()s are enabled (via the C way: by not defining NDEBUG), and
some routines do additional sanity checks inside "#ifdef Py_DEBUG"
blocks.
---------------------------------------------------------------------------
COUNT_ALLOCS introduced in 0.9.9
partly broken in 2.2 and 2.2.1
Each type object grows three new members:
/* Number of times an object of this type was allocated. */
int tp_allocs;
/* Number of times an object of this type was deallocated. */
int tp_frees;
/* Highwater mark: the maximum value of tp_allocs - tp_frees so
* far; or, IOW, the largest number of objects of this type alive at
* the same time.
*/
int tp_maxalloc;
Allocation and deallocation code keeps these counts up to date.
Py_Finalize() displays a summary of the info returned by sys.getcounts()
(see below), along with assorted other special allocation counts (like
the number of tuple allocations satisfied by a tuple free-list, the number
of 1-character strings allocated, etc).
Before Python 2.2, type objects were immortal, and the COUNT_ALLOCS
implementation relies on that. As of Python 2.2, heap-allocated type/
class objects can go away. COUNT_ALLOCS can blow up in 2.2 and 2.2.1
because of this; this was fixed in 2.2.2. Use of COUNT_ALLOCS makes
all heap-allocated type objects immortal, except for those for which no
object of that type is ever allocated.
Starting with Python 2.3, If Py_TRACE_REFS is also defined, COUNT_ALLOCS
arranges to ensure that the type object for each allocated object
appears in the doubly-linked list of all objects maintained by
Py_TRACE_REFS.
Special gimmicks:
sys.getcounts()
Return a list of 4-tuples, one entry for each type object for which
at least one object of that type was allocated. Each tuple is of
the form:
(tp_name, tp_allocs, tp_frees, tp_maxalloc)
Each distinct type object gets a distinct entry in this list, even
if two or more type objects have the same tp_name (in which case
there's no way to distinguish them by looking at this list). The
list is ordered by time of first object allocation: the type object
for which the first allocation of an object of that type occurred
most recently is at the front of the list.
---------------------------------------------------------------------------
LLTRACE introduced well before 1.0
Compile in support of Low Level TRACE-ing of the main interpreter loop.
When this preprocessor symbol is defined, before eval_frame
(eval_code2 before 2.2) executes a frame's code it checks the frame's
global namespace for a variable "__lltrace__". If such a variable is
found, mounds of information about what the interpreter is doing are
sprayed to stdout, such as every opcode and opcode argument and values
pushed onto and popped off the value stack.
Not useful very often, but very useful when needed.
---------------------------------------------------------------------------
CALL_PROFILE introduced for Python 2.3
Count the number of function calls executed.
When this symbol is defined, the ceval mainloop and helper functions
count the number of function calls made. It keeps detailed statistics
about what kind of object was called and whether the call hit any of
the special fast paths in the code.
--- NEW FILE: pymemcompat.h ---
/* The idea of this file is that you bundle it with your extension,
#include it, program to Python 2.3's memory API and have your
extension build with any version of Python from 1.5.2 through to
2.3 (and hopefully beyond). */
#ifndef Py_PYMEMCOMPAT_H
#define Py_PYMEMCOMPAT_H
#include "Python.h"
/* There are three "families" of memory API: the "raw memory", "object
memory" and "object" families. (This is ignoring the matter of the
cycle collector, about which more is said below).
Raw Memory:
PyMem_Malloc, PyMem_Realloc, PyMem_Free
Object Memory:
PyObject_Malloc, PyObject_Realloc, PyObject_Free
Object:
PyObject_New, PyObject_NewVar, PyObject_Del
The raw memory and object memory allocators both mimic the
malloc/realloc/free interface from ANSI C, but the object memory
allocator can (and, since 2.3, does by default) use a different
allocation strategy biased towards lots of "small" allocations.
The object family is used for allocating Python objects, and the
initializers take care of some basic initialization (setting the
refcount to 1 and filling out the ob_type field) as well as having
a somewhat different interface.
Do not mix the families! E.g. do not allocate memory with
PyMem_Malloc and free it with PyObject_Free. You may get away with
it quite a lot of the time, but there *are* scenarios where this
will break. You Have Been Warned.
Also, in many versions of Python there are an insane amount of
memory interfaces to choose from. Use the ones described above. */
#if PY_VERSION_HEX < 0x01060000
/* raw memory interface already present */
/* there is no object memory interface in 1.5.2 */
#define PyObject_Malloc PyMem_Malloc
#define PyObject_Realloc PyMem_Realloc
#define PyObject_Free PyMem_Free
/* the object interface is there, but the names have changed */
#define PyObject_New PyObject_NEW
#define PyObject_NewVar PyObject_NEW_VAR
#define PyObject_Del PyMem_Free
#endif
/* If your object is a container you probably want to support the
cycle collector, which was new in Python 2.0.
Unfortunately, the interface to the collector that was present in
Python 2.0 and 2.1 proved to be tricky to use, and so changed in
2.2 -- in a way that can't easily be papered over with macros.
This file contains macros that let you program to the 2.2 GC API.
Your module will compile against any Python since version 1.5.2,
but the type will only participate in the GC in versions 2.2 and
up. Some work is still necessary on your part to only fill out the
tp_traverse and tp_clear fields when they exist and set tp_flags
appropriately.
It is possible to support both the 2.0 and 2.2 GC APIs, but it's
not pretty and this comment block is too narrow to contain a
desciption of what's required... */
#if PY_VERSION_HEX < 0x020200B1
#define PyObject_GC_New PyObject_New
#define PyObject_GC_NewVar PyObject_NewVar
#define PyObject_GC_Del PyObject_Del
#define PyObject_GC_Track(op)
#define PyObject_GC_UnTrack(op)
#endif
#endif /* !Py_PYMEMCOMPAT_H */
_______________________________________________
Stackless-checkins mailing list
Stackless-checkins at stackless.com
http://www.stackless.com/mailman/listinfo/stackless-checkins
- Previous message: [Stackless-checkins] CVS: slpdev/src/2.2/src/Mac/scripts BuildApplet.icns, NONE, 1.1 BuildApplet.plist, NONE, 1.1 buildpkg.py, NONE, 1.1 cachersrc.py, NONE, 1.1 genallsuites.py, NONE, 1.1 mkestrres-macerrors.h, NONE, 1.1
- Next message: [Stackless-checkins] CVS: slpdev/src/2.2/src/Misc/RPM python-2.3.spec, NONE, 1.1
- Messages sorted by:
[ date ]
[ thread ]
[ subject ]
[ author ]
More information about the Stackless-checkins
mailing list