PyBloqs is a data visualisation framework for Python, designed to create interactive reports. It works with your favourite plotting library - be it Plotly, Matplotlib, Bokeh or Highcharts - to produce ‘blocks’ of data, which contain and present text, tables or images, all of which can be styled using CSS. These blocks can be manipulated and presented together to form reports, which in turn can be exported and viewed in several formats, including HTML and PDF.

Nowadays, reports are boring; everyone wants to create a responsive dashboard in the cloud. But business reality is that quite often, point-in-time reports are a required solution. On top of simplifying report creation, Pybloqs can help with data analysis by visualising grids of mixed data like plots and tables.

The framework is designed to be used offline in productionised code, but it works just as well in a Jupyter notebook, where it enhances the layout options. All blocks render in-line within this system, allowing for quick analysis during research. All output is based on creating HTML, which can be output into many formats, providing both simple maintenance and fungibility. Because PyBlogs supports interactive plotting libraries, users can create reports with a high degree of visual appeal and interactivity.

In this article, we will be exploring the capability of PyBloqs using open-source data on the fuel consumption of cars, retrieved from Corgis-Edu. The aim of this is to demonstrate how the framework can be used to display and enhance data for a variety of purposes. If you would like to follow along with this guide, or utilise PyBloqs similarly, we recommend following the quickstart guide found on the PyBloqs GitHub page here.

Upon first impressions, the information we are using is unintuitive to work with, as it comes packaged as a Python list of dicts. Whilst this might be useful for reference, it isn’t visually interesting, does not allow us to derive insights at a glance, and can be confusing to inexperienced users.

To remedy this, we can create visual aids in the form of graphs and charts. To start with, we will load the Corgis dataset into a Pandas dataframe using the following:

import cars
import copy

flattened_cars = []
for single_car in cars.get_car():
    single_car = copy.deepcopy(single_car)
    engine_statistics = single_car['Engine Information'].pop('Engine Statistics')
    
    flattened_cars.append({
        **single_car['Identification'],
        **single_car['Fuel Information'],
        **single_car['Engine Information'],
        **engine_statistics,
        **single_car['Dimensions'],
    })
import pandas  as pd
df = pd.DataFrame.from_records(flattened_cars)
df

This outputs the following. Again, this data is open-source, and we are using it as an illustrative example:

In this case we would like to create a report using some of the data.

The limitation of this is that, whilst they can create very useful displays of data, these graphs still need to be exported and manually organised in presentations or reports. This is where PyBloqs comes in. Using the framework, we can enter a few parameters to instruct the system to automate the exporting process.

First, we will create a plot using this Highcharts AP I :

from pybloqs import Block
from pybloqs.plot import
Chart, Column, Plot, Series, XAxis, YAxis
title_text = "Fuel types as a percentage of available cars by year"
title = Block(title=title_text)
year_fuel_count = df.value_counts(["Fuel Type", "Year"]).unstack(level=-1).fillna(0).astype(int
)
year_fuel_count_plot = Plot(
    year_fuel_count.T, 
    Column(stacking="percent"), 
    Chart(zoom_type="xy"),
    XAxis(title=dict (text="Year")), 
    YAxis(title=dict (text="Percentage")),
    title=title_text,
)
year_fuel_count_plot

The Highcharts API used is a Python wrapper on the official JS Highcharts API, as there is no official Python version. This outputs the following chart:

In this example, we have taken the data in the table above, and used PyBloqs to create an interactive bar chart. In this use-case, the information is clearly laid out, and we can hover over the bars or select different categories to drill-down into the information, or tailor it towards different audiences.

As Plotly is a Python library, we just wrap the fig output with a pybloqs.Block

import plotly.express as px

fig = px.scatter(df, x="Horsepower", y="City mpg", title="Horsepower - City mpg")
hp_vs_mpg_plot = Block(fig)

This creates a plot that looks like this:

The advantage of an output like this is that it allows the end user to identify specific outliers from complex data at a glance: all that is required is to hover over a specific point. The user can also utilise any of the additional data analysis tools offered by a Plotly output, zooming in, comparing and isolating specific data points.

Whilst this is useful, there is something else to consider. Reports don't just have plots: they also normally contain tabular data. PyBloqs makes it easy to manage the formatting of tabular data that is sourced from a pd.DataFrame. For example, we are easily able to add date, heatmap, and conditional formatters to make our table more visually informative:

import pybloqs.block.colors as colors
from pybloqs import HTMLJinjaTableBlock
pybloqs.block.table_formatter_builder import CommonTableFormatterBuilder

top = df.groupby("Make").head(1)
top = top.loc[top.Make.str.match("Mercedes$|BMW$|Volkswagen$|Mitsubishi$|Toyota$|Honda$|GMC$|Chevorlet|Buick")]
top = top.reset_index().drop(["index", "ID"], axis=1).sort_values(by=["Torque"])
fmt = (
    CommonTableFormatterBuilder(top, table_width=None, font_size=14)
    .date_columns(columns=["Year"])
    .hide_columns(columns=["Make", "Hybrid", "Fuel Type", "Engine Type", "Transmission"])
    .heatmap(columns=["Horsepower"], min_color=colors.GREEN, max_color=colors.HEATMAP_RED)
    .divider_line_vertical(column="Horsepower")
    .threshold(column="Torque", threshold_column="Horsepower")
    .color_background_conditionally_matching(value="All-wheel drive", color=colors.BLUE)
    .color_background_conditionally(condition=Lambdav: v > 5, color=colors.YELLOW, columns=["Number of Forward Gears"])
)
table_block = HTMLJinjaTableBlock(top, formatters=fmt.formatters, use_default_formatters=False, title="Cars with high horsepower")
table_block

This code block outputs the following:

As seen in the code, we can decouple the formatting from the underlying data, such as hiding irrelevant columns. We also apply the builder pattern to streamline the syntax.

However, a few disparate plots and tables do not make a report. To finalise, we can assemble the plots together into a report and render it as an interactive HTML document. PyBloqs also supports rendering to PDF - this outputs the HTML as static images and searchable text within a file.

from pybloqs import Grid

report = Grid([year_fuel_count_plot, hp_vs_mpg_plot, table_block], cols=2)
report.show()
report.save("Car report.html")

And the output can be found here: car report.html

As you can see, this report is pretty simple. The beauty of PyBloqs is in its convenience. Each block of data can be manipulated and output in different ways, depending on what is required. A complex report containing pages of analysis for staff engineers can be created using the same framework as a simple, bare-bones document for presentation to a management committee. Really, the limitation is your own creativity.

We hope that this quick run through has piqued your interest. If you’d like to explore using PyBloqs yourself, please see the documentation here:

https://pybloqs.readthedocs.io/en/latest/

https://github.com/man-group/PyBloqs

If you have any questions, please reach out to us here. https://twitter.com/ManQuantTech

 

I am interested in other Tech Articles.

To receive e-mail alerts whenever new Tech Articles or Events are posted on this site, please subscribe below.

Subscribe

 

Find out more about Technology at Man Group

Important information

This information is communicated and/or distributed by the relevant Man entity identified below (collectively the "Company") subject to the following conditions and restriction in their respective jurisdictions.

Opinions expressed are those of the author and may not be shared by all personnel of Man Group plc (‘Man’). These opinions are subject to change without notice, are for information purposes only and do not constitute an offer or invitation to make an investment in any financial instrument or in any product to which the Company and/or its affiliates provides investment advisory or any other financial services. Any organisations, financial instrument or products described in this material are mentioned for reference purposes only which should not be considered a recommendation for their purchase or sale. Neither the Company nor the authors shall be liable to any person for any action taken on the basis of the information provided. Some statements contained in this material concerning goals, strategies, outlook or other non-historical matters may be forward-looking statements and are based on current indicators and expectations. These forward-looking statements speak only as of the date on which they are made, and the Company undertakes no obligation to update or revise any forward-looking statements. These forward-looking statements are subject to risks and uncertainties that may cause actual results to differ materially from those contained in the statements. The Company and/or its affiliates may or may not have a position in any financial instrument mentioned and may or may not be actively trading in any such securities. Unless stated otherwise all information is provided by the Company. Past performance is not indicative of future results.

Unless stated otherwise this information is communicated by the relevant entity listed below.

Australia: To the extent this material is distributed in Australia it is communicated by Man Investments Australia Limited ABN 47 002 747 480 AFSL 240581, which is regulated by the Australian Securities & Investments Commission ('ASIC'). This information has been prepared without taking into account anyone’s objectives, financial situation or needs.

Austria/Germany/Liechtenstein: To the extent this material is distributed in Austria, Germany and/or Liechtenstein it is communicated by Man (Europe) AG, which is authorised and regulated by the Liechtenstein Financial Market Authority (FMA). Man (Europe) AG is registered in the Principality of Liechtenstein no. FL-0002.420.371-2. Man (Europe) AG is an associated participant in the investor compensation scheme, which is operated by the Deposit Guarantee and Investor Compensation Foundation PCC (FL-0002.039.614-1) and corresponds with EU law. Further information is available on the Foundation's website under www.eas-liechtenstein.li. This material is of a promotional nature.

European Economic Area: Unless indicated otherwise this material is communicated in the European Economic Area by Man Asset Management (Ireland) Limited (‘MAMIL’) which is registered in Ireland under company number 250493 and has its registered office at 70 Sir John Rogerson's Quay, Grand Canal Dock, Dublin 2, Ireland. MAMIL is authorised and regulated by the Central Bank of Ireland under number C22513.

Hong Kong SAR: To the extent this material is distributed in Hong Kong SAR, this material is communicated by Man Investments (Hong Kong) Limited and has not been reviewed by the Securities and Futures Commission in Hong Kong. This material can only be communicated to intermediaries, and professional clients who are within one of the professional investors exemptions contained in the Securities and Futures Ordinance and must not be relied upon by any other person(s).

Japan: To the extent this material is distributed in Japan it is communicated by Man Group Japan Limited, Financial Instruments Business Operator, Director of Kanto Local Finance Bureau (Financial instruments firms) No. 624 for the purpose of providing information on investment strategies, investment services, etc. provided by Man Group, and is not a disclosure document based on laws and regulations. This material can only be communicated only to professional investors (i.e. specific investors or institutional investors as defined under Financial Instruments Exchange Law) who may have sufficient knowledge and experience of related risks.

Switzerland: To the extent the material is distributed in Switzerland the communicating entity is Man Investments AG, Huobstrasse 3, 8808 Pfäffikon SZ, Switzerland. Man Investment AG is regulated by the Swiss Financial Market Supervisory Authority ('FINMA').

United Kingdom: Unless indicated otherwise this material is communicated in the United Kingdom by Man Solutions Limited (‘MSL’) which is an investment company as defined in section 833 of the Companies Act 2006. MSL is registered in England and Wales under number 3385362 and has its registered office at Riverbank House, 2 Swan Lane, London, EC4R 3AD, United Kingdom. MSL is authorised and regulated by the UK Financial Conduct Authority (the ‘FCA’) under number 185637.

United States: To the extent this material is distributed in the United States, it is communicated and distributed by Man Investments, Inc. (‘Man Investments’). Man Investments is registered as a broker-dealer with the SEC and is a member of the Financial Industry Regulatory Authority (‘FINRA’). Man Investments is also a member of the Securities Investor Protection Corporation (‘SIPC’). Man Investments is a wholly owned subsidiary of Man Group plc. The registration and memberships described above in no way imply a certain level of skill or expertise or that the SEC, FINRA or the SIPC have endorsed Man Investments. Man Investments, 452 Fifth Avenue, 27th fl., New York, NY 10018.

This material is proprietary information and may not be reproduced or otherwise disseminated in whole or in part without prior written consent. Any data services and information available from public sources used in the creation of this material are believed to be reliable. However accuracy is not warranted or guaranteed. © Man 2022

MKT003833/ST/GL/W

Please update your browser

Unfortunately we no longer support Internet Explorer 8, 7 and older for security reasons.

Please update your browser to a later version and try to access our site again.

Many thanks.