Pliant forum message

Concerning release: 33
Subject: Contribution/Question
Title: Critique of my first meta program (evar.pli)
initial message posted by John Eikenberry on 2000/03/13 12:25:59
After much toiling away I finally realized I was making things more difficult 
than necessary. So I went for the simple/direct approach. Below then is my try at 
creating a meta function that handles wrapping an (C library) external global 
variable. It seems to work in simple tests (included), but I haven't had the 
chance to try it in a real application yet.

I'm curious if this seems the correct and appropriate way to form this meta
function, and any comments or critiques on how I did it. 

Of particular interest is the way I tried to not pollute the namespace to much 
by using all the underscores in my naming (like __func__ below). Is there a 
better way to not pollute the namespace when doing something like this.

----- evar.pli -----
module "/pliant/language/compiler.pli"

named_expression ext_var
  function FUNC_VAR_NAME
    EFUN_BODY
  gvar VAR_PNT_TYPE VAR_NAME :> (the_function FUNC_VAR_NAME):executable map VAR_TYPE
  function DA_FUNC
    VAR_NAME :> (the_function FUNC_VAR_NAME):executable map VAR_TYPE
  gvar DelayedAction DA_VAR
  DA_VAR function :> (the_function DA_FUNC)
  pliant_restore_actions append addressof:DA_VAR

meta evar e
  if e:size<>3
    return
  if e:2:0:size<>2
    return

  var Link:Expression var_decl :> expression duplicate ext_var
  var_decl substitute "EFUN_BODY" e:2:0
  var_decl substitute "VAR_NAME" e:1
  var_decl substitute "FUNC_VAR_NAME" (expression ident "__func__"+e:1:ident)
  var_decl substitute "DA_FUNC" (expression ident "__da_func__"+e:1:ident)
  var_decl substitute "DA_VAR" (expression ident "__da_var__"+e:1:ident)
  var_decl substitute "VAR_TYPE" e:0
  var_decl substitute "VAR_PNT_TYPE" ((expression immediat (Pointer:VAR_TYPE)) substitute "VAR_TYPE" e:0)
  var_decl near e
  e compile_as var_decl

# here's an example interface to a couple global variable in the slang library
evar Int sltt_screen_rows
  external slib "SLtt_Screen_Rows"

evar Int sltt_screen_cols
  external slib "SLtt_Screen_Cols"

console sltt_screen_rows eol
sltt_screen_rows := 10
console sltt_screen_rows eol

console sltt_screen_cols eol
sltt_screen_cols := 10
console sltt_screen_cols eol
answer posted by Hubert Tonneau on 2000/03/13 13:38:49
Really nice stuff indead !

> Of particular interest is the way I tried to not pollute the namespace to much 
> by using all the underscores in my naming (like __func__ below). Is there a 
> better way to not pollute the namespace when doing something like this.named_expression ext_var

Use names ending with an underscore, then call 'auto_rename' method:

 named_expression ext_var
   function FUNC_VAR_NAME_
     EFUN_BODY
   gvar VAR_PNT_TYPE VAR_NAME :> (the_function FUNC_VAR_NAME_):executable map VAR_TYPE
   function DA_FUNC_
     VAR_NAME :> (the_function FUNC_VAR_NAME_):executable map VAR_TYPE
   gvar DelayedAction DA_VAR_
   DA_VAR function :> the_function DA_FUNC_
   pliant_restore_actions append addressof:DA_VAR_

 meta evar e
   if e:size<>3
     return
   if e:2:0:size<>2
     return

   var Link:Expression var_decl :> expression duplicate ext_var
   var_decl substitute "EFUN_BODY" e:2:0
   var_decl substitute "VAR_NAME" e:1
   var_decl auto_rename
   var_decl substitute "VAR_TYPE" e:0
   var_decl substitute "VAR_PNT_TYPE" ((expression immediat Pointer:VAR_TYPE) substitute "VAR_TYPE" e:0)
   var_decl near e
   e compile_as var_decl
answer posted by John Eikenberry on 2000/03/13 14:33:36
Thanks for the vote of confidence... and for the auto_rename tip. Very handy.

As a side note to this, I think the enumerated type would make a good addition
to the meta programming samples page. I found it very helpful in working on this, 
and there isn't an example of creating a new type dynamically on it yet.

Thanks again.

One last thing. For anyone who is thinking of using the above. Hubert's rewrite
(with auto_rename) is what I'm using, but he missed one trailing underscore on 
this line:

   DA_VAR function :> the_function DA_FUNC_

Is should be:

   DA_VAR_ function :> the_function DA_FUNC_
answer posted by Hubert Tonneau on 2000/03/13 15:01:56
A few more remarks about your program:

- the prototype of a delayed action function recorded in
  'pliant_restore_actions' list must be:

  function xxx parameter filehandle
    arg Address parameter ; arg Int filehandle

  Also you don't care of both 'parameter' and 'filehandle' arguments.
  If you don't do that, your program will crash if I switch the calling
  convension from registers to stack only.

- When you access an argument, you must have check that it exists:

  if e:2:0:size<>2 # will crash is e:2 has no arguments
     return
 
  Should be:

  if e:2:size<>1 or e:2:0:size<>2
    return

- adding this would improve the error checking capabilities of you meta:

  if e:2:0:ident<>"external"
    return

- In your notation, you specify two times that the variable is external:
  through the 'e' of 'evar' and through the 'external' keyword. I'd rather
  use:

  evar Int sltt_screen_rows slib "SLtt_Screen_Rows"

  or

  var Int sltt_screen_rows
    external slib "SLtt_Screen_Rows"

  I prefer the first one because it's more straight forward
  (see implementation at end), but both could be valid.

- Another purely cosmetical comment:

  var_decl near e

  instruction is probably not needed because 'compile_as' method will do it.


module "/pliant/language/compiler.pli"

named_expression ext_var
  function external_function_
    external DLL ID
  gvar Pointer:TYPE NAME :> (the_function external_function_):executable map TYPE
  function restore_ parameter filehandle
    arg Address parameter ; arg Int filehandle
    NAME :> (the_function external_function_):executable map TYPE
  gvar DelayedAction da_
  da_ function :> the_function restore_  Address Int
  pliant_restore_actions append addressof:da_

meta evar e
  if e:size<>4 or (e:0 constant Type)=null or e:1:ident="" or not (e:2 cast Str) or not (e:3 cast Str)
    return
  var Link:Expression decl :> expression duplicate ext_var
  decl substitute "TYPE" e:0
  decl substitute "NAME" e:1
  decl substitute "DLL" e:2
  decl substitute "ID" e:3
  decl auto_rename
  e compile_as decl

export evar
answer posted by Hubert Tonneau on 2000/03/13 15:24:37
Also I have not tested that it really works, this could be a more
straight forward implementation:

module "/pliant/language/compiler.pli"

named_expression ext_var
  function external_function_
    external DLL ID
  function NAME -> v
    arg_RW TYPE v
    v :> (the_function external_function_):executable map TYPE

meta evar e
  if e:size<>4 or (e:0 constant Type)=null or e:1:ident="" or not (e:2 cast Str) or not (e:3 cast Str)
    return
  e compile_as (expression duplicate ext_var auto_rename substitute TYPE e:0 substitute NAME e:1 substitute DLL e:2 substitute ID e:3)

export evar
answer posted by John Eikenberry on 2000/03/14 02:16:02
Thanks for the comments... here's a few questions in return. ;)

I wasn't sure about the DelayedAction part and I wasn't sure how to test it.
Is there a way to explicitally test delayed actions? Related to this, in the
version in your last reply there isn't a DelayedAction. I assume you thinking 
that by defining the argument as arg_RW that the DA isn't needed? I'll play with
this tonight.

> - In your notation, you specify two times that the variable is external:
>   through the 'e' of 'evar' and through the 'external' keyword.

Hmm... I was shooting for a syntax similar to the external function syntax. 
Which is why I had the redundant 'external' in there. Also I think this makes
the external nature of the variable more explicit. So I think I'm leaning 
toward the 2nd.

I'll mess around with some variations and repost what I finally decide upon.
answer posted by Hubert Tonneau on 2000/03/14 04:14:27
> Is there a way to explicitally test delayed actions?

Put a 'console' instruction in it.

> Related to this, in the
> version in your last reply there isn't a DelayedAction.

In my last post, there is no more DelayedAction since the variable is
a function instead, so extracting the function address will be done each
time the variable is used, whereas with your implementation (the new symbol
is really a variable), it must be updated on .dump reload since the
executable address of the external function (so the address of the
external variable in facts) may have changed.

It is one import, and carefully designed part of Pliant syntax: a variable
is noted the same way as a function with no argument, so when you use
an identifyer, it can be any of them.

My program would be buggy if the constants evaluation algorithm in
/pliant/language/optimizer/consteval.pli
decides to evaluate the expression (the_function external_function_):executable
at compile time. It is not the case, because if you extend my program
with the following extra checkings, it will still compile:

module "/pliant/language/compiler.pli"

meta must_not_be_constant e
  if e:size=2
    var Pointer:Type t :> (e:0 constant Type) map Type
    if addressof:t<>null and (e:1 constant t)=null
      e set_void_result
      # if we succeed to compile this it means that the second argument
      # cannot be a constant with the type of the first argument
    
named_expression ext_var
  function external_function_
    external DLL ID
  function NAME -> v
    arg_RW TYPE v
    v :> (the_function external_function_):executable map TYPE
  must_not_be_constant Address (the_function external_function_):executable

meta evar e
  if e:size<>4 or (e:0 constant Type)=null or e:1:ident="" or not (e:2 cast Str) or not (e:3 cast Str)
    return
  e compile_as (expression duplicate ext_var auto_rename substitute TYPE e:0 substitute NAME e:1 substitute DLL e:2 substitute ID e:3)

> Also I think this make the external nature of the variable more explicit.
> So I think I'm leaning toward the 2nd.

Ok, you can do that, with the following extra constains:
- you should still implement the external variable as a function, just as
  I did , because if you implement it as a pointer, nothing will prevent
  a bad behaved program to change the pointer using the ':>' instruction.
- since you choose the more complex syntax, you have to carefully check
  all arguments, not only the ones that are meaningfull for you meta, so
  that your meta does not conflict with other ones. You need things like:
  if e:size<>3
     return
  if e:2:size<>1 or e:2:ident<>"{}"
    return
  if e:2:0:size<>2 or e:2:0:ident<>"external"
    return
  ...
answer posted by John Eikenberry on 2000/03/15 06:58:32
> named_expression ext_var
>   function external_function_
>     external DLL ID
>   function NAME -> v
>     arg_RW TYPE v
>     v :> (the_function external_function_):executable map TYPE

Using this method, how do you assign a new value to the variable? It doesn't seem
like it would be possible to me? Unless there is something I don't understand
about how the arg_RW works. But based on my testing, it doesn't work. Ie. I get 
an error when I try to assign a value to the global variable (like I did in the
example in my original post). Here's the error:

Failed to compile :=   (Int c  ?)
  compile file:evar.pli (internals) 36 18
  compile file:evar.pli (internals) 36 1
  compile file:evar.pli (internals) 36 1
  parse file:evar.pli (internals) 37 1
  module file:evar.pli
answer posted by Hubert Tonneau on 2000/03/15 15:12:45
> Failed to compile :=   (Int c  ?)

This error message says that ':=' could not compile because it's
first argument is a constant integer.

There is nothing wrong about this meta, but something is wrong
elsewhere (the prototype of the 'executable' field in the
'Function' data type is wrong, and the constants evaluation
algorithm is also wrong because it will sometime turn a read write
variable to a constant).
I will investigate this futher in order to correct it before release 34.

In the meantime, you can make your program work through inserting
a 'has_side_effects' instruction that will prevent constants evaluation.

named_expression ext_var
  function external_function_
    external DLL ID
  function NAME -> v
    arg_RW TYPE v
    has_side_effects
    v :> (the_function external_function_):executable map TYPE
answer posted by John Eikenberry on 2000/03/16 12:15:52
As promised, here is my (current) final version on evar.pli. Of course, based on
the current discussions on casting, this may change. Anyways... here it is:

--- evar.pli ---

module "/pliant/language/compiler.pli"

named_expression ext_var
  function external_function_
    EFUN_BODY
  function NAME -> v
    arg_RW TYPE v
    has_side_effects
    v :> (the_function external_function_):executable map TYPE

meta var e
  if e:size<>3 or (e:0 constant Type)=null
    return
  if e:2:size<>1 or e:2:ident<>"{}"
    return
  if e:2:0:size<>2 or e:2:0:ident<>"external"
    return    
    
  var Link:Expression var_decl :> expression duplicate ext_var
  var_decl substitute "TYPE" e:0
  var_decl substitute "NAME" e:1
  var_decl substitute "EFUN_BODY" e:2:0
  var_decl auto_rename
  e compile_as var_decl

export var

------

You'll notice this is a little different in format than Hubert's similar version.
That's because I prefer the substitutes broken up over multiple lines. I think
this makes what's going on more obvious and keeps any one line from getting too 
long (as Pliant doesn't have line wrapping).
answer posted by Patrice Ossona de Mendez on 2000/03/16 13:40:44
"I prefer the substitutes broken up over multiple lines. I think
 this makes what's going on more obvious and keeps any one line from getting too 
 long (as Pliant doesn't have line wrapping)."

That's the reason why I posted some meta in the forum of 1999/03, like "unwrap":

unwrap expression 
  duplicate ext_var 
  auto_rename 
  substitute TYPE e:0 
  substitute NAME e:1 
  substitute DLL e:2 
  substitute ID e:3

does exactly what we think...
answer posted by Hubert Tonneau on 2000/03/16 14:29:41
The 'evar.pli' will be included in release 34 main tree
as /pliant/language/compiler/misc/evar.pli module.
Also this module will not be a submodule of the general /pliant/language/compiler.pli
module, so you'll have to include it explicitely.
This is because it provides a not wildly used, but also very usefull feature.

If somebody posts a well tested 'CChar' implementation, it could also be
included.

Patrice, could you post a new 'unwrap' implementation, that does not rely
on other 'unbloc' and so on stuff, so that I can include it in release 34
as the best preposed alternative for line wrapping.
Should go in /pliant/language/basic/unwrap.pli module.
answer posted by John Eikenberry on 2000/03/16 14:31:02
When first starting with Pliant (ie. not that long ago) I tried messing around 
with unwrap, conjonction, etc. They didn't work with the new version and I didn't 
have the experience to fix them. Now that I have a bit of meta programming
experience, mabey I'll give it another whirl.

Thanks for reminding me. :)
answer posted by Patrice Ossona de Mendez on 2000/03/16 15:59:44
I propose the following that seems to work (the original version was really 
bugy!)


module "/pliant/language/compiler.pli"
# unwrap:
#
# unwrap (toto ('{}' (a b c) (d e) (f g h i) j)) -> (toto a b c d e f g h i j)
#
meta unwrap e
  if not (not e:is_pure_ident and e:size=2 and e:0:is_pure_ident and e:1:ident="{}")
    return
  var Link:Expression e2 :> (expression immediat (toto X) ident e:0:ident)
  for (var Int i) 0 e:1:size-1
    if e:1:i:is_pure_ident
      e2 insert "X" (expression immediat Ident ident e:1:i:ident near e)
      for (var Int j) 0 e:1:i:size-1
       e2 insert "X" e:1:i:j
    eif e:1:i:ident = "()"
      for (var Int j) 0 e:1:i:size-1
       e2 insert "X" e:1:i:j
    else
      e2 insert "X" e:1:i
  e2 remove "X"
  e compile_as e2
answer posted by Patrice Ossona de Mendez on 2000/03/16 15:59:56
I propose the following that seems to work (the original version was really 
bugy!)


module "/pliant/language/compiler.pli"
# unwrap:
#
# unwrap (toto ('{}' (a b c) (d e) (f g h i) j)) -> (toto a b c d e f g h i j)
#
meta unwrap e
  if not (not e:is_pure_ident and e:size=2 and e:0:is_pure_ident and e:1:ident="{}")
    return
  var Link:Expression e2 :> (expression immediat (toto X) ident e:0:ident)
  for (var Int i) 0 e:1:size-1
    if e:1:i:is_pure_ident
      e2 insert "X" (expression immediat Ident ident e:1:i:ident near e)
      for (var Int j) 0 e:1:i:size-1
       e2 insert "X" e:1:i:j
    eif e:1:i:ident = "()"
      for (var Int j) 0 e:1:i:size-1
       e2 insert "X" e:1:i:j
    else
      e2 insert "X" e:1:i
  e2 remove "X"
  e compile_as e2