Visual Studio Code - How to display formatted Jupyter stack trace from "full output data"

CorneliusRoemer
March 1, 2023
128 views
0 votes
2 Answers

I am debugging Python code in VS Code using the Jupyter integration in interactive mode.

The stack trace I get is only partially displayed, because "Output exceeds the size limit." I would like to see the full stack trace.

I’m advised to "Open the full output data in a text editor".

When I do that, by clicking on "in a text editor" I get something that looks like JSON but is too messy for human consumption:

{
    "name": "InvalidOperationError",
    "message": "window expression not allowed in aggregation",
    "stack": "u001b[0;31m---------------------------------------------------------------------------u001b[0mnu001b[0;31mInvalidOperationErroru001b[0m                     Traceback (most recent call last)nu001b[1;32m/Users/corneliusromer/Downloads/tmp/so.pyu001b[0m in u001b[0;36mline 2nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=38'>39</a>u001b[0m u001b[39m#%%u001b[39;00mnu001b[0;32m----> <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=39'>40</a>u001b[0m dfu001b[39m.u001b[39;49mwith_columns(nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=40'>41</a>u001b[0m     [nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=41'>42</a>u001b[0m         plu001b[39m.u001b[39;49mstruct(nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=42'>43</a>u001b[0m             [nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=43'>44</a>u001b[0m                 (nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=44'>45</a>u001b[0m                     plu001b[39m.u001b[39;49mcol(specs[specnm][u001b[39m"u001b[39;49mu001b[39myvaru001b[39;49mu001b[39m"u001b[39;49m])nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=45'>46</a>u001b[0m                     u001b[39m-u001b[39;49m plu001b[39m.u001b[39;49mcol(specs[specnm][u001b[39m"u001b[39;49mu001b[39myvaru001b[39;49mu001b[39m"u001b[39;49m])u001b[39m.u001b[39;49mmean()u001b[39m.u001b[39;49mover(specs[specnm][u001b[39m"u001b[39;49mu001b[39mgvarsu001b[39;49mu001b[39m"u001b[39;49m])nu001b[1;32m      <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=46'>47</a>u001b[0m                 )u001b[39m.u001b[39;49mabs(),nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=47'>48</a>u001b[0m                 u001b[39m*u001b[39;49mspecs[specnm][u001b[39m"u001b[39;49mu001b[39mxvarsu001b[39;49mu001b[39m"u001b[39;49m],nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=48'>49</a>u001b[0m             ]nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=49'>50</a>u001b[0m         )nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=50'>51</a>u001b[0m         u001b[39m.u001b[39;49mapply(nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=51'>52</a>u001b[0m             partial(nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=52'>53</a>u001b[0m                 ols_fitted, yvaru001b[39m=u001b[39;49mspecs[specnm][u001b[39m"u001b[39;49mu001b[39myvaru001b[39;49mu001b[39m"u001b[39;49m], xvarsu001b[39m=u001b[39;49mspecs[specnm][u001b[39m"u001b[39;49mu001b[39mxvarsu001b[39;49mu001b[39m"u001b[39;49m]nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=53'>54</a>u001b[0m             )nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=54'>55</a>u001b[0m         )nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=55'>56</a>u001b[0m         u001b[39m.u001b[39;49mover([u001b[39m"u001b[39;49mu001b[39mdateu001b[39;49mu001b[39m"u001b[39;49m, u001b[39m"u001b[39;49mu001b[39midu001b[39;49mu001b[39m"u001b[39;49m])nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=56'>57</a>u001b[0m         u001b[39m.u001b[39;49malias(u001b[39mfu001b[39;49mu001b[39m"u001b[39;49mu001b[39mfitted_u001b[39;49mu001b[39m{u001b[39;49;00mspecnmu001b[39m}u001b[39;49;00mu001b[39m"u001b[39;49m)nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=57'>58</a>u001b[0m         u001b[39mforu001b[39;49;00m specnm u001b[39minu001b[39;49;00m u001b[39mlistu001b[39;49m(specsu001b[39m.u001b[39;49mkeys())nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=58'>59</a>u001b[0m     ]nu001b[1;32m     <a href='file:///Users/corneliusromer/Downloads/tmp/so.py?line=59'>60</a>u001b[0m )nnFile u001b[0;32m/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py:6139u001b[0m, in u001b[0;36mDataFrame.with_columnsu001b[0;34m(self, exprs, *more_exprs, **named_exprs)u001b[0mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5985'>5986</a>u001b[0m u001b[39mdefu001b[39;00m u001b[39mwith_columnsu001b[39m(nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5986'>5987</a>u001b[0m     u001b[39mselfu001b[39m,nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5987'>5988</a>u001b[0m     exprs: IntoExpr u001b[39m|u001b[39m Iterable[IntoExpr] u001b[39m=u001b[39m u001b[39mNoneu001b[39;00m,nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5988'>5989</a>u001b[0m     u001b[39m*u001b[39mmore_exprs: IntoExpr,nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5989'>5990</a>u001b[0m     u001b[39m*u001b[39mu001b[39m*u001b[39mnamed_exprs: IntoExpr,nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5990'>5991</a>u001b[0m ) u001b[39m-u001b[39mu001b[39m>u001b[39m Self:nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5991'>5992</a>u001b[0m u001b[39m    u001b[39mu001b[39m"""u001b[39;00mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5992'>5993</a>u001b[0m u001b[39m    Add columns to this DataFrame.u001b[39;00mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=5993'>5994</a>u001b[0m nu001b[0;32m   (...)u001b[0mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6133'>6134</a>u001b[0m nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6134'>6135</a>u001b[0m u001b[39m    """u001b[39;00mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6135'>6136</a>u001b[0m     u001b[39mreturnu001b[39;00m u001b[39mselfu001b[39mu001b[39m.u001b[39m_from_pydf(nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6136'>6137</a>u001b[0m         u001b[39mselfu001b[39;49mu001b[39m.u001b[39;49mlazy()nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6137'>6138</a>u001b[0m         u001b[39m.u001b[39;49mwith_columns(exprs, u001b[39m*u001b[39;49mmore_exprs, u001b[39m*u001b[39;49mu001b[39m*u001b[39;49mnamed_exprs)nu001b[0;32m-> <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6138'>6139</a>u001b[0m         u001b[39m.u001b[39;49mcollect(no_optimizationu001b[39m=u001b[39;49mu001b[39mTrueu001b[39;49;00m)nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6139'>6140</a>u001b[0m         u001b[39m.u001b[39m_dfnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py?line=6140'>6141</a>u001b[0m     )nnFile u001b[0;32m/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py:1323u001b[0m, in u001b[0;36mLazyFrame.collectu001b[0;34m(self, type_coercion, predicate_pushdown, projection_pushdown, simplify_expression, no_optimization, slice_pushdown, common_subplan_elimination, streaming)u001b[0mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1311'>1312</a>u001b[0m     common_subplan_elimination u001b[39m=u001b[39m u001b[39mFalseu001b[39;00mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1313'>1314</a>u001b[0m ldf u001b[39m=u001b[39m u001b[39mselfu001b[39mu001b[39m.u001b[39m_ldfu001b[39m.u001b[39moptimization_toggle(nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1314'>1315</a>u001b[0m     type_coercion,nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1315'>1316</a>u001b[0m     predicate_pushdown,nu001b[0;32m   (...)u001b[0mnu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1320'>1321</a>u001b[0m     streaming,nu001b[1;32m   <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1321'>1322</a>u001b[0m )nu001b[0;32m-> <a href='file:///opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py?line=1322'>1323</a>u001b[0m u001b[39mreturnu001b[39;00m pliu001b[39m.u001b[39mwrap_df(ldfu001b[39m.u001b[39;49mcollect())nnu001b[0;31mInvalidOperationErroru001b[0m: window expression not allowed in aggregation"
}

How can I display the stack variable in a nicely formatted manner, similar to how the excerpt is shown in the image below?

Is there maybe a VS code extension for this? Or does the value of the stack key look like some known formatting? Is this using ANSI escapes or something else that I could use a CLI tool for viewing?

Answers

Chosen as BEST ANSWER
- CorneliusRoemer
- March 1, 2023 at 7:29 pm
- 0 votes
0
This appears to be a known issue.

The terminal knows how to format the stack string correctly. So one can copy the whole thing and paste it into the terminal, prefixing with echo -e.

Or one save the file and runs the following to automate the JSON value extraction:
```
jj stack <tmp.json | less -R
```
The result is still not ideal and could benefit from some post-processing, filtering out the long <a href>s but it's better than the unformatted:

The advantage of using less is that lines don't necessarily wrap. If you use plain terminal output, the long lines will wrap making things unreadable.

One can also increase the value of notebook.output.textLineLimit in VS Code settings.

(Edit)

A more concise traceback, often sufficient, can be obtained by wrapping the raising code in a try/except and printing the stack trace using traceback, e.g.:

import traceback
try:
    [].pop()  # replace this line with your erroring code
except:
    traceback.print_exc()

For the example given above, the stack trace becomes much more readable:

Traceback (most recent call last):
  File "<ipython-input-6-ce1246cd962c>", line 4, in <module>
    df.with_columns(
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/dataframe/frame.py", line 6139, in with_columns
    .collect(no_optimization=True)
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Caskroom/miniforge/base/envs/py11/lib/python3.11/site-packages/polars/internals/lazyframe/frame.py", line 1323, in collect
    return pli.wrap_df(ldf.collect())
                       ^^^^^^^^^^^^^
exceptions.InvalidOperationError: window expression not allowed in aggregation

Please signup or login to give your own answer.

Click here to cancel reply.

Visual Studio Code – How to display formatted Jupyter stack trace from "full output data"

Answers