I converted a Pandas dataframe to an HTML output using the DataFrame.to_html
function. When I save this to a separate HTML file, the file shows truncated output.
For example, in my TEXT column,
df.head(1)
will show
The film was an excellent effort...
instead of
The film was an excellent effort in deconstructing the complex social sentiments that prevailed during this period.
This rendition is fine in the case of a screen-friendly format of a massive Pandas dataframe, but I need an HTML file that will show complete tabular data contained in the dataframe, that is, something that will show the latter text element rather than the former text snippet.
How would I be able to show the complete, non-truncated text data for each element in my TEXT column in the HTML version of the information? I would imagine that the HTML table would have to display long cells to show the complete data, but as far as I understand, only column-width parameters can be passed into the DataFrame.to_html
function.
Set the display.max_colwidth
option to None
(or -1
before version 1.0):
pd.set_option('display.max_colwidth', None)
For example, in IPython, we see that the information is truncated to 50 characters. Anything in excess is ellipsized:
https://i.stack.imgur.com/hANGS.png
If you set the display.max_colwidth
option, the information will be displayed fully:
https://i.stack.imgur.com/Nxg2q.png
pd.set_option('display.max_columns', None)
id
(second argument) can fully show the columns.
max_colwidth
can solve the truncate issue caused by a field has too long values. However, I think the truncate issues for most people is actually too many columns. So, this max_columns
should be the accepted one.
While pd.set_option('display.max_columns', None)
sets the number of the maximum columns shown, the option pd.set_option('display.max_colwidth', -1)
sets the maximum width of each single field.
For my purposes I wrote a small helper function to fully print huge data frames without affecting the rest of the code. It also reformats float numbers and sets the virtual display width. You may adopt it for your use cases.
def print_full(x):
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.width', 2000)
pd.set_option('display.float_format', '{:20,.2f}'.format)
pd.set_option('display.max_colwidth', None)
print(x)
pd.reset_option('display.max_rows')
pd.reset_option('display.max_columns')
pd.reset_option('display.width')
pd.reset_option('display.float_format')
pd.reset_option('display.max_colwidth')
display.width
is the missing ingredient here. Thanks.
with pd.option_context(...): display(x)
Jupyter Users
Whenever I need this for just one cell, I use this:
with pd.option_context('display.max_colwidth', None):
display(df)
None
can also lead to performance issues in Notebooks.
Try this too:
pd.set_option("max_columns", None) # show all cols
pd.set_option('max_colwidth', None) # show full width of showing cols
pd.set_option("expand_frame_repr", False) # print cols side by side as it's supposed to be
pd.set_option("display.max_columns", None)
when you meet Pattern matched multiple keys
error.
Another way of viewing the full content of the cells in a Pandas dataframe is to use IPython's display functions:
from IPython.display import HTML
HTML(df.to_html())
The following code results in the error below:
pd.set_option('display.max_colwidth', -1)
FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width.
Instead, use:
pd.set_option('display.max_colwidth', None)
This accomplishes the task and complies with versions of Pandas following version 1.0.
For those looking to do this in Dask:
I could not find a similar option in Dask, but if I simply do this in same notebook for Pandas it works for Dask too.
import pandas as pd
import dask.dataframe as dd
pd.set_option('display.max_colwidth', -1) # This will set the no truncate for Pandas as well as for Dask. I am not sure how it does for Dask though, but it works.
train_data = dd.read_csv('./data/train.csv')
train_data.head(5)
For those who like to reduce typing (i.e., everyone!): pd.set_option('max_colwidth', None)
does the same thing
Display the full dataframe for a specific cell:
import pandas as pd
with pd.option_context('display.max_colwidth', None,
'display.max_columns', None,
'display.max_rows', None):
display(df)
The method above can be extended with more options.
Updated helper function from Karl Adler:
def display_full(x):
with pd.option_context('display.max_rows', None,
'display.max_columns', None,
'display.width', 2000,
'display.float_format', '{:20,.2f}'.format,
'display.max_colwidth', None):
display(x)
Change display options for all cells:
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_rows', None)
pd.set_option('display.max_columns', None)
display(df)
Success story sharing
None
to mean unlimited.max_columns
answer worked for me, which usesNone
as the second argument ofset_option
.with pd.option_context('display.max_colwidth', -1): display(df)
display.max_colwidth
to-1
I got aFutureWarning
. Replacing the-1
withNone
worked, and eliminated the warning.