Conference turris::debug

1857.0. "How to find the stack corrupter" by COL01::VSEMUSCHIN (Duck and Recover !) Thu Apr 10 1997 07:08

    Hi !
    
    I'm trying to debug for customer very big DECwindows application.
    It looks like, that there one of subroutines started an asynchronous
    action giving it as parameter (IOSB?) the volatile address on the
    stack. When later another program is called (always the same program)
    it's stack becomes corrupted (always the same varibale). This problem
    is present on Alpha's with both 6.2 and 7.1 versions of OpenVMS.
    
    Now I try using debugger to find out which routines operate on the
    same stack depth. I know, that watchpoint in stack is imopossible.
    I already tried to protect stack pages, but the only routine I catched
    was ... image activator.
    
    I want to use following construction:
    
    I know, that varibales that get hurts are among ADDR1 and ADDR2 in
    stack (ADDR1 > variables > ADDR2) , so I will try
    
    set break/call when( %SP >= ADDR2 ) ! I assume break/call will freeze the
                                        ! execution before entrying the called pgm
        do ( step/into; if (%SP >= ADDR2 ) then (show stack 1) else (go) )
    
    Any other ideas ?
    (The construction above doesn't work because on some syntactical
    problems - see the previous note),
    
    =Seva

    !set language fortran
    !set radix hexadecimal
    !set break/call when( %SP .ge. 7EE43520 ) -
    !    do(  step/into; -                                                 
    !         if (%FP .le. 7EE43520).and.(%FP.ge.40000000)
    !         then (show stack 1; examine %FP; show symbol/address * in 0;)-
    !         else (go ); -
    !    );
    !

Also, see notes 1304 and 1533, which are denoted by 
the keyword, STACK.

Mary