[Stackless] Proposal: new tasklet method set_args(*args, **kw) that combines setup() and remove()

Anselm Kruis a.kruis at science-computing.de
Tue Dec 3 09:54:38 CET 2013


Hi,

I'm fully aware that bind(), setup() and other methods are an ad-hoc 
API. But it does not prevent us from making it consistent and exposing 
the building blocks. This helps people to improve existing code and to 
understand the inner working of stackless.

On the other hand your objections about the missing composability are 
also valid. A new, well designed API would be a clear benefit. Again we 
have to consider timing and our limited resources. Improving the 
existing API is a low hanging fruit; we can implement those changes even 
for 2.7.6-slp.

Designing and correctly (production quality) implementing a new API is a 
demanding and time consuming project. Development should be decoupled 
from the regular maintenance of 2.7-slp.

Cheers
   Anselm


Am 02.12.2013 23:02, schrieb Christian Tismer:
> Hi,
>
> bind() and setup() were my lazy approach to split the initialization
> of a tasklet without the need to fiddle with argument tuples.
> Unfortunately, nobody objected or proposed a better (but maybe more
> complex)
> approach.
>
> I am thinking if it is good to keep these old functions and extend them,
> or if it
> was nicer to use totally new names and keep the old names, but deprecate
> them?
>
> There are many old ad-hoc decisions which were also a bad combination of
> the old round-robin scheduler model, which nowadays becomes quite
> questionable.
>
> Would it not be better to make a completely new API, instead of
> extending the old one,
> which pretends that the old one was good? It was just simple, but if we
> extend it
> further, it is still a limited scheduler with _one_ round-robin
> principle, but no longer simple,
> and therefore "bad", no?
>
> Just a thought: Would it not be better to leave that alone and start
> over, by breaking
> things down to the real building blocks, where the current API is just a
> special case?
> Going a bit further, I am back to the "composable" feature that was
> implemented
> in PyPy's stackless.
>
> Ok, simplifying again, the one approach does not exclude the other.
> Extending bind/setup
> is a way to improve the current, limited model. At the same time I would
> like to see that
> as an arcane layout which is one of many possibles, and we could express
> it as the
> default model which can co-exist with others.
> (sorry, the topic is actually too complex to cover in this thread. Maybe
> you need to
> read on PyPy's stackless features to get a feeling about composability)
>
> http://doc.pypy.org/en/latest/stackless.html#theory-of-composability
>
> cheers - chris
>
>
> On 21/11/13 10:54, Kristján Valur Jónsson wrote:
>> Sounds fine.
>> Btw, I'd argue with a different set_args()
>> I always found it weird to have separate 'bind' and 'setup'
>>
>> How about extending "bind" like this:
>>
>> def bind(self, function, args=None, keywords=None):
>> The presence of args and/or keywords as being non-None, would then
>> imply a setup, without scheduling it.
>>
>> K
>>
>>> -----Original Message-----
>>> From: stackless-bounces at stackless.com [mailto:stackless-
>>> bounces at stackless.com] On Behalf Of Anselm Kruis
>>> Sent: 19. nóvember 2013 11:16
>>> To: stackless at stackless.com
>>> Subject: Re: [Stackless] Proposal: new tasklet method set_args(*args,
>>> **kw)
>>> that combines setup() and remove()
>>>
>>> Hi Kristján,
>>>
>>> I wasn't aware of stackless.atomic(). It's not yet documented.
>>>
>>> Obviously there are a few pending issues before we can release
>>> 2.7.6-slp.
>>>
>>> - add set_args()
>>> - add run_remove()
>>> - update the documentation
>>> - update the changelog
>>>
>>> Anything else? We should really try to create a consistent and well
>>> documented release.
>>>
>>> Cheers
>>>     Anselm
>>>
>>>
>>> Am 19.11.2013 10:16, schrieb Kristján Valur Jónsson:
>>>> It does, inasmuch that the implicit "insert" that  exists on many
>>>> methods is
>>> indeed annoying.
>>>> But it is not really a problem, because:
>>>>
>>>> def set_args(t, args=(), kw={}):
>>>>     with stackless.atomic():
>>>>       t.setup(*args, **kw):
>>>>       t.remove()
>>>>
>>>>
>>>> A more annoying problem that isn't solvable, and that is that there
>>>> is no
>>> run_remove(), i.e. no way switch to a tasklet and remove the caller
>>> from the
>>> run queue.
>>>> This is something that is needed in stacklesslib.async:
>>>>
>>>>
>>>> def call_async(callable, args=(), kwargs={}):
>>>>       awaiter = Awaiter(stackless.getcurrent())
>>>>       callee = stackless.tasklet(async_call_helper)
>>>>       future = futures.Future()
>>>>       with atomic():
>>>>           callee(future, awaiter, callable, args, kwargs)
>>>>           try:
>>>>               # here, a run(remove=True) or a switch() primitive
>>>> would be useful
>>>>               callee.run()
>>>>           finally:
>>>>               # need this here, in case caller gets awoken by other
>>>> means, e.g.
>>> exception
>>>>               awaiter.caller_continued = True
>>>>       return future
>>>>
>>>> def async_call_helper(future, awaiter, callable, args, kwargs):
>>>>       # remove the caller from the runnables queue.  There is a
>>>> window here
>>> where other tasklets
>>>>       # might run, we need perhaps a primitive to perform this task
>>>>       try:
>>>>           awaiter.caller.remove()
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: stackless-bounces at stackless.com [mailto:stackless-
>>>>> bounces at stackless.com] On Behalf Of Anselm Kruis
>>>>> Sent: 18. nóvember 2013 10:39
>>>>> To: stackless at stackless.com
>>>>> Subject: [Stackless] Proposal: new tasklet method set_args(*args,
>>>>> **kw) that combines setup() and remove()
>>>>>
>>>>> Hi,
>>>>>
>>>>> I propose to add a new method set_args(*args, **kw) to class tasklet,
>>>>> that combines
>>>>>
>>>>>         stackless.setup(*args, **kw)
>>>>>         stackless.remove()
>>>>>
>>>>> Rationale: it is currently not possible to create an alive tasklet
>>>>> without scheduling it (except via unpickling or direct __setstate__).
>>>>>
>>>>> With the new bind_thread() method, one can think of use cases where
>>>>> one thread creates tasklets and another thread executes them. No need
>>>>> to insert these tasklets into the current run queue. It could even
>>>>> cause
>>> races.
>>>>> With set_args() in place, setup() would become a simple shortcut for
>>>>> set_args() followed by insert().
>>>>>
>>>>> Does this proposal make sense?
>>>>>
>>>>> regards
>>>>>      Anselm
>>>>>
>>>>> --
>>>>>     Dipl. Phys. Anselm Kruis                       science +
>>>>> computing ag
>>>>>     Senior Solution Architect                      Ingolstädter
>>>>> Str. 22
>>>>>     email A.Kruis at science-computing.de<mailto:A.Kruis at science-
>>> computing.de>             80807 München, Germany
>>>>>     phone +49 89 356386 874  fax 737               www.science-
>>> computing.de<http://www.science-computing.de>
>>>>> --
>>>>> Vorstandsvorsitzender/Chairman of the board of management:
>>>>> Gerd-Lothar Leonhart
>>>>> Vorstand/Board of Management:
>>>>> Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr. Ingrid
>>>>> Zech Vorsitzender des Aufsichtsrats/ Chairman of the Supervisory
>>>>> Board:
>>>>> Philippe Miltin
>>>>> Sitz/Registered Office: Tuebingen
>>>>> Registergericht/Registration Court: Stuttgart
>>>>> Registernummer/Commercial Register No.: HRB 382196
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Stackless mailing list
>>>>> Stackless at stackless.com<mailto:Stackless at stackless.com>
>>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>
>>>>
>>>>
>>>> It does, inasmuch that the implicit "insert" that  exists on many
>>>> methods is indeed annoying.
>>>> But it is not really a problem, because:
>>>> def set_args(t, args=(), kw={}):
>>>>     with stackless.atomic():
>>>>       t.setup(*args, **kw):
>>>>       t.remove()
>>>> A more annoying problem that isn't solvable, and that is that there is
>>>> no run_remove(), i.e. no way switch to a tasklet and remove the caller
>>>> from the run queue.
>>>> This is something that is needed in stacklesslib.async:
>>>> def call_async(callable, args=(), kwargs={}):
>>>>       awaiter = Awaiter(stackless.getcurrent())
>>>>       callee = stackless.tasklet(async_call_helper)
>>>>       future = futures.Future()
>>>>       with atomic():
>>>>           callee(future, awaiter, callable, args, kwargs)
>>>>           try:
>>>> # here, a run(remove=True) or a switch() primitive would be useful
>>>>               callee.run()
>>>>           finally:
>>>>               # need this here, in case caller gets awoken by other
>>>> means, e.g. exception
>>>>               awaiter.caller_continued = True
>>>>       return future
>>>> def async_call_helper(future, awaiter, callable, args, kwargs):
>>>>       # remove the caller from the runnables queue.  There is a window
>>>> here where other tasklets
>>>>       # might run, we need perhaps a primitive to perform this task
>>>>       try:
>>>> awaiter.caller.remove()
>>>>> -----Original Message-----
>>>>> From: stackless-bounces at stackless.com [mailto:stackless-
>>>>> bounces at stackless.com] On Behalf Of Anselm Kruis
>>>>> Sent: 18. nóvember 2013 10:39
>>>>> To: stackless at stackless.com
>>>>> Subject: [Stackless] Proposal: new tasklet method set_args(*args,
>>>>> **kw) that combines setup() and remove()
>>>>>
>>>>> Hi,
>>>>>
>>>>> I propose to add a new method set_args(*args, **kw) to class tasklet,
>>>>> that combines
>>>>>
>>>>>         stackless.setup(*args, **kw)
>>>>>         stackless.remove()
>>>>>
>>>>> Rationale: it is currently not possible to create an alive tasklet
>>>>> without scheduling it (except via unpickling or direct __setstate__).
>>>>>
>>>>> With the new bind_thread() method, one can think of use cases where
>>>>> one thread creates tasklets and another thread executes them. No need
>>>>> to insert these tasklets into the current run queue. It could even
>>>>> cause
>>> races.
>>>>> With set_args() in place, setup() would become a simple shortcut for
>>>>> set_args() followed by insert().
>>>>>
>>>>> Does this proposal make sense?
>>>>>
>>>>> regards
>>>>>     Anselm
>>>>>
>>>>> --
>>>>>    Dipl. Phys. Anselm Kruis                       science +
>>>>> computing ag
>>>>>    Senior Solution Architect                      Ingolstädter Str. 22
>>>>>    emailA.Kruis at science-computing.de
>>>> <mailto:A.Kruis at science-computing.de>             80807 München,
>>>> Germany
>>>>>    phone +49 89 356386 874  fax 737www.science-computing.de
>>>>> <http://www.science-computing.de>
>>>>> --
>>>>> Vorstandsvorsitzender/Chairman of the board of management:
>>>>> Gerd-Lothar Leonhart
>>>>> Vorstand/Board of Management:
>>>>> Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr. Ingrid
>>>>> Zech Vorsitzender des Aufsichtsrats/ Chairman of the Supervisory
>>>>> Board:
>>>>> Philippe Miltin
>>>>> Sitz/Registered Office: Tuebingen
>>>>> Registergericht/Registration Court: Stuttgart
>>>>> Registernummer/Commercial Register No.: HRB 382196
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Stackless mailing list
>>>>> Stackless at stackless.com <mailto:Stackless at stackless.com>
>>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>
>>>> _______________________________________________
>>>> Stackless mailing list
>>>> Stackless at stackless.com
>>>> http://www.stackless.com/mailman/listinfo/stackless
>>>>
>>> --
>>>    Dipl. Phys. Anselm Kruis                       science + computing ag
>>>    Senior Solution Architect                      Ingolstädter Str. 22
>>>    email A.Kruis at science-computing.de             80807 München, Germany
>>>    phone +49 89 356386 874  fax 737
>>> www.science-computing.de
>>> --
>>> Vorstandsvorsitzender/Chairman of the board of management:
>>> Gerd-Lothar Leonhart
>>> Vorstand/Board of Management:
>>> Dr. Bernd Finkbeiner, Michael Heinrichs, Dr. Arno Steitz, Dr. Ingrid
>>> Zech
>>> Vorsitzender des Aufsichtsrats/ Chairman of the Supervisory Board:
>>> Philippe Miltin
>>> Sitz/Registered Office: Tuebingen
>>> Registergericht/Registration Court: Stuttgart Registernummer/Commercial
>>> Register No.: HRB 382196
>>>
>>>
>>> _______________________________________________
>>> Stackless mailing list
>>> Stackless at stackless.com
>>> http://www.stackless.com/mailman/listinfo/stackless
>>>
>>
>>
>> _______________________________________________
>> Stackless mailing list
>> Stackless at stackless.com
>> http://www.stackless.com/mailman/listinfo/stackless
>
>

-- 
  Dipl. Phys. Anselm Kruis                       science + computing ag
  Senior Solution Architect                      Ingolstädter Str. 22
  email A.Kruis at science-computing.de             80807 München, Germany
  phone +49 89 356386 874  fax 737               www.science-computing.de
-- 
Vorstandsvorsitzender/Chairman of the board of management:
Gerd-Lothar Leonhart
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Michael Heinrichs, 
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196




More information about the Stackless mailing list