Adds visualization for the demo.
These are the file necessary to display the visualization for
the Cobalt demo. It consists of
(a) The Google Visualization library in third_party
(b) Some Python scripts and an html file under tools/demo
Change-Id: Ie20b1be7999d2b72efd0b12306ccda488e237e4b
diff --git a/third_party/__init__.py b/third_party/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/third_party/__init__.py
diff --git a/third_party/google_visualization/.gitignore b/third_party/google_visualization/.gitignore
new file mode 100644
index 0000000..0d20b64
--- /dev/null
+++ b/third_party/google_visualization/.gitignore
@@ -0,0 +1 @@
+*.pyc
diff --git a/third_party/google_visualization/COPYRIGHT b/third_party/google_visualization/COPYRIGHT
new file mode 100644
index 0000000..d645695
--- /dev/null
+++ b/third_party/google_visualization/COPYRIGHT
@@ -0,0 +1,202 @@
+
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
diff --git a/third_party/google_visualization/README b/third_party/google_visualization/README
new file mode 100644
index 0000000..5a73854
--- /dev/null
+++ b/third_party/google_visualization/README
@@ -0,0 +1,25 @@
+Copyright (C) 2009 Google Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Installing the library:
+ python ./setup.py install
+ (You might need root privileges to do this)
+
+Testing the library:
+ python ./setup.py test
+
+Dependencies:
+ On Python <2.6 you will need to have simplejson[1] installed on your system.
+
+ [1]: http://pypi.python.org/pypi/simplejson/
diff --git a/third_party/google_visualization/__init__.py b/third_party/google_visualization/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/third_party/google_visualization/__init__.py
diff --git a/third_party/google_visualization/gviz_api.py b/third_party/google_visualization/gviz_api.py
new file mode 100755
index 0000000..99245b8
--- /dev/null
+++ b/third_party/google_visualization/gviz_api.py
@@ -0,0 +1,1091 @@
+#!/usr/bin/python
+#
+# Copyright (C) 2009 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Converts Python data into data for Google Visualization API clients.
+
+This library can be used to create a google.visualization.DataTable usable by
+visualizations built on the Google Visualization API. Output formats are raw
+JSON, JSON response, JavaScript, CSV, and HTML table.
+
+See http://code.google.com/apis/visualization/ for documentation on the
+Google Visualization API.
+"""
+
+__author__ = "Amit Weinstein, Misha Seltzer, Jacob Baskin"
+
+import cgi
+import cStringIO
+import csv
+import datetime
+try:
+ import json
+except ImportError:
+ import simplejson as json
+import types
+
+
+class DataTableException(Exception):
+ """The general exception object thrown by DataTable."""
+ pass
+
+
+class DataTableJSONEncoder(json.JSONEncoder):
+ """JSON encoder that handles date/time/datetime objects correctly."""
+
+ def __init__(self):
+ json.JSONEncoder.__init__(self,
+ separators=(",", ":"),
+ ensure_ascii=False)
+
+ def default(self, o):
+ if isinstance(o, datetime.datetime):
+ if o.microsecond == 0:
+ # If the time doesn't have ms-resolution, leave it out to keep
+ # things smaller.
+ return "Date(%d,%d,%d,%d,%d,%d)" % (
+ o.year, o.month - 1, o.day, o.hour, o.minute, o.second)
+ else:
+ return "Date(%d,%d,%d,%d,%d,%d,%d)" % (
+ o.year, o.month - 1, o.day, o.hour, o.minute, o.second,
+ o.microsecond / 1000)
+ elif isinstance(o, datetime.date):
+ return "Date(%d,%d,%d)" % (o.year, o.month - 1, o.day)
+ elif isinstance(o, datetime.time):
+ return [o.hour, o.minute, o.second]
+ else:
+ return super(DataTableJSONEncoder, self).default(o)
+
+
+class DataTable(object):
+ """Wraps the data to convert to a Google Visualization API DataTable.
+
+ Create this object, populate it with data, then call one of the ToJS...
+ methods to return a string representation of the data in the format described.
+
+ You can clear all data from the object to reuse it, but you cannot clear
+ individual cells, rows, or columns. You also cannot modify the table schema
+ specified in the class constructor.
+
+ You can add new data one or more rows at a time. All data added to an
+ instantiated DataTable must conform to the schema passed in to __init__().
+
+ You can reorder the columns in the output table, and also specify row sorting
+ order by column. The default column order is according to the original
+ table_description parameter. Default row sort order is ascending, by column
+ 1 values. For a dictionary, we sort the keys for order.
+
+ The data and the table_description are closely tied, as described here:
+
+ The table schema is defined in the class constructor's table_description
+ parameter. The user defines each column using a tuple of
+ (id[, type[, label[, custom_properties]]]). The default value for type is
+ string, label is the same as ID if not specified, and custom properties is
+ an empty dictionary if not specified.
+
+ table_description is a dictionary or list, containing one or more column
+ descriptor tuples, nested dictionaries, and lists. Each dictionary key, list
+ element, or dictionary element must eventually be defined as
+ a column description tuple. Here's an example of a dictionary where the key
+ is a tuple, and the value is a list of two tuples:
+ {('a', 'number'): [('b', 'number'), ('c', 'string')]}
+
+ This flexibility in data entry enables you to build and manipulate your data
+ in a Python structure that makes sense for your program.
+
+ Add data to the table using the same nested design as the table's
+ table_description, replacing column descriptor tuples with cell data, and
+ each row is an element in the top level collection. This will be a bit
+ clearer after you look at the following examples showing the
+ table_description, matching data, and the resulting table:
+
+ Columns as list of tuples [col1, col2, col3]
+ table_description: [('a', 'number'), ('b', 'string')]
+ AppendData( [[1, 'z'], [2, 'w'], [4, 'o'], [5, 'k']] )
+ Table:
+ a b <--- these are column ids/labels
+ 1 z
+ 2 w
+ 4 o
+ 5 k
+
+ Dictionary of columns, where key is a column, and value is a list of
+ columns {col1: [col2, col3]}
+ table_description: {('a', 'number'): [('b', 'number'), ('c', 'string')]}
+ AppendData( data: {1: [2, 'z'], 3: [4, 'w']}
+ Table:
+ a b c
+ 1 2 z
+ 3 4 w
+
+ Dictionary where key is a column, and the value is itself a dictionary of
+ columns {col1: {col2, col3}}
+ table_description: {('a', 'number'): {'b': 'number', 'c': 'string'}}
+ AppendData( data: {1: {'b': 2, 'c': 'z'}, 3: {'b': 4, 'c': 'w'}}
+ Table:
+ a b c
+ 1 2 z
+ 3 4 w
+ """
+
+ def __init__(self, table_description, data=None, custom_properties=None):
+ """Initialize the data table from a table schema and (optionally) data.
+
+ See the class documentation for more information on table schema and data
+ values.
+
+ Args:
+ table_description: A table schema, following one of the formats described
+ in TableDescriptionParser(). Schemas describe the
+ column names, data types, and labels. See
+ TableDescriptionParser() for acceptable formats.
+ data: Optional. If given, fills the table with the given data. The data
+ structure must be consistent with schema in table_description. See
+ the class documentation for more information on acceptable data. You
+ can add data later by calling AppendData().
+ custom_properties: Optional. A dictionary from string to string that
+ goes into the table's custom properties. This can be
+ later changed by changing self.custom_properties.
+
+ Raises:
+ DataTableException: Raised if the data and the description did not match,
+ or did not use the supported formats.
+ """
+ self.__columns = self.TableDescriptionParser(table_description)
+ self.__data = []
+ self.custom_properties = {}
+ if custom_properties is not None:
+ self.custom_properties = custom_properties
+ if data:
+ self.LoadData(data)
+
+ @staticmethod
+ def CoerceValue(value, value_type):
+ """Coerces a single value into the type expected for its column.
+
+ Internal helper method.
+
+ Args:
+ value: The value which should be converted
+ value_type: One of "string", "number", "boolean", "date", "datetime" or
+ "timeofday".
+
+ Returns:
+ An item of the Python type appropriate to the given value_type. Strings
+ are also converted to Unicode using UTF-8 encoding if necessary.
+ If a tuple is given, it should be in one of the following forms:
+ - (value, formatted value)
+ - (value, formatted value, custom properties)
+ where the formatted value is a string, and custom properties is a
+ dictionary of the custom properties for this cell.
+ To specify custom properties without specifying formatted value, one can
+ pass None as the formatted value.
+ One can also have a null-valued cell with formatted value and/or custom
+ properties by specifying None for the value.
+ This method ignores the custom properties except for checking that it is a
+ dictionary. The custom properties are handled in the ToJSon and ToJSCode
+ methods.
+ The real type of the given value is not strictly checked. For example,
+ any type can be used for string - as we simply take its str( ) and for
+ boolean value we just check "if value".
+ Examples:
+ CoerceValue(None, "string") returns None
+ CoerceValue((5, "5$"), "number") returns (5, "5$")
+ CoerceValue(100, "string") returns "100"
+ CoerceValue(0, "boolean") returns False
+
+ Raises:
+ DataTableException: The value and type did not match in a not-recoverable
+ way, for example given value 'abc' for type 'number'.
+ """
+ if isinstance(value, tuple):
+ # In case of a tuple, we run the same function on the value itself and
+ # add the formatted value.
+ if (len(value) not in [2, 3] or
+ (len(value) == 3 and not isinstance(value[2], dict))):
+ raise DataTableException("Wrong format for value and formatting - %s." %
+ str(value))
+ if not isinstance(value[1], types.StringTypes + (types.NoneType,)):
+ raise DataTableException("Formatted value is not string, given %s." %
+ type(value[1]))
+ js_value = DataTable.CoerceValue(value[0], value_type)
+ return (js_value,) + value[1:]
+
+ t_value = type(value)
+ if value is None:
+ return value
+ if value_type == "boolean":
+ return bool(value)
+
+ elif value_type == "number":
+ if isinstance(value, (int, long, float)):
+ return value
+ raise DataTableException("Wrong type %s when expected number" % t_value)
+
+ elif value_type == "string":
+ if isinstance(value, unicode):
+ return value
+ else:
+ return str(value).decode("utf-8")
+
+ elif value_type == "date":
+ if isinstance(value, datetime.datetime):
+ return datetime.date(value.year, value.month, value.day)
+ elif isinstance(value, datetime.date):
+ return value
+ else:
+ raise DataTableException("Wrong type %s when expected date" % t_value)
+
+ elif value_type == "timeofday":
+ if isinstance(value, datetime.datetime):
+ return datetime.time(value.hour, value.minute, value.second)
+ elif isinstance(value, datetime.time):
+ return value
+ else:
+ raise DataTableException("Wrong type %s when expected time" % t_value)
+
+ elif value_type == "datetime":
+ if isinstance(value, datetime.datetime):
+ return value
+ else:
+ raise DataTableException("Wrong type %s when expected datetime" %
+ t_value)
+ # If we got here, it means the given value_type was not one of the
+ # supported types.
+ raise DataTableException("Unsupported type %s" % value_type)
+
+ @staticmethod
+ def EscapeForJSCode(encoder, value):
+ if value is None:
+ return "null"
+ elif isinstance(value, datetime.datetime):
+ if value.microsecond == 0:
+ # If it's not ms-resolution, leave that out to save space.
+ return "new Date(%d,%d,%d,%d,%d,%d)" % (value.year,
+ value.month - 1, # To match JS
+ value.day,
+ value.hour,
+ value.minute,
+ value.second)
+ else:
+ return "new Date(%d,%d,%d,%d,%d,%d,%d)" % (value.year,
+ value.month - 1, # match JS
+ value.day,
+ value.hour,
+ value.minute,
+ value.second,
+ value.microsecond / 1000)
+ elif isinstance(value, datetime.date):
+ return "new Date(%d,%d,%d)" % (value.year, value.month - 1, value.day)
+ else:
+ return encoder.encode(value)
+
+ @staticmethod
+ def ToString(value):
+ if value is None:
+ return "(empty)"
+ elif isinstance(value, (datetime.datetime,
+ datetime.date,
+ datetime.time)):
+ return str(value)
+ elif isinstance(value, unicode):
+ return value
+ elif isinstance(value, bool):
+ return str(value).lower()
+ else:
+ return str(value).decode("utf-8")
+
+ @staticmethod
+ def ColumnTypeParser(description):
+ """Parses a single column description. Internal helper method.
+
+ Args:
+ description: a column description in the possible formats:
+ 'id'
+ ('id',)
+ ('id', 'type')
+ ('id', 'type', 'label')
+ ('id', 'type', 'label', {'custom_prop1': 'custom_val1'})
+ Returns:
+ Dictionary with the following keys: id, label, type, and
+ custom_properties where:
+ - If label not given, it equals the id.
+ - If type not given, string is used by default.
+ - If custom properties are not given, an empty dictionary is used by
+ default.
+
+ Raises:
+ DataTableException: The column description did not match the RE, or
+ unsupported type was passed.
+ """
+ if not description:
+ raise DataTableException("Description error: empty description given")
+
+ if not isinstance(description, (types.StringTypes, tuple)):
+ raise DataTableException("Description error: expected either string or "
+ "tuple, got %s." % type(description))
+
+ if isinstance(description, types.StringTypes):
+ description = (description,)
+
+ # According to the tuple's length, we fill the keys
+ # We verify everything is of type string
+ for elem in description[:3]:
+ if not isinstance(elem, types.StringTypes):
+ raise DataTableException("Description error: expected tuple of "
+ "strings, current element of type %s." %
+ type(elem))
+ desc_dict = {"id": description[0],
+ "label": description[0],
+ "type": "string",
+ "custom_properties": {}}
+ if len(description) > 1:
+ desc_dict["type"] = description[1].lower()
+ if len(description) > 2:
+ desc_dict["label"] = description[2]
+ if len(description) > 3:
+ if not isinstance(description[3], dict):
+ raise DataTableException("Description error: expected custom "
+ "properties of type dict, current element "
+ "of type %s." % type(description[3]))
+ desc_dict["custom_properties"] = description[3]
+ if len(description) > 4:
+ raise DataTableException("Description error: tuple of length > 4")
+ if desc_dict["type"] not in ["string", "number", "boolean",
+ "date", "datetime", "timeofday"]:
+ raise DataTableException(
+ "Description error: unsupported type '%s'" % desc_dict["type"])
+ return desc_dict
+
+ @staticmethod
+ def TableDescriptionParser(table_description, depth=0):
+ """Parses the table_description object for internal use.
+
+ Parses the user-submitted table description into an internal format used
+ by the Python DataTable class. Returns the flat list of parsed columns.
+
+ Args:
+ table_description: A description of the table which should comply
+ with one of the formats described below.
+ depth: Optional. The depth of the first level in the current description.
+ Used by recursive calls to this function.
+
+ Returns:
+ List of columns, where each column represented by a dictionary with the
+ keys: id, label, type, depth, container which means the following:
+ - id: the id of the column
+ - name: The name of the column
+ - type: The datatype of the elements in this column. Allowed types are
+ described in ColumnTypeParser().
+ - depth: The depth of this column in the table description
+ - container: 'dict', 'iter' or 'scalar' for parsing the format easily.
+ - custom_properties: The custom properties for this column.
+ The returned description is flattened regardless of how it was given.
+
+ Raises:
+ DataTableException: Error in a column description or in the description
+ structure.
+
+ Examples:
+ A column description can be of the following forms:
+ 'id'
+ ('id',)
+ ('id', 'type')
+ ('id', 'type', 'label')
+ ('id', 'type', 'label', {'custom_prop1': 'custom_val1'})
+ or as a dictionary:
+ 'id': 'type'
+ 'id': ('type',)
+ 'id': ('type', 'label')
+ 'id': ('type', 'label', {'custom_prop1': 'custom_val1'})
+ If the type is not specified, we treat it as string.
+ If no specific label is given, the label is simply the id.
+ If no custom properties are given, we use an empty dictionary.
+
+ input: [('a', 'date'), ('b', 'timeofday', 'b', {'foo': 'bar'})]
+ output: [{'id': 'a', 'label': 'a', 'type': 'date',
+ 'depth': 0, 'container': 'iter', 'custom_properties': {}},
+ {'id': 'b', 'label': 'b', 'type': 'timeofday',
+ 'depth': 0, 'container': 'iter',
+ 'custom_properties': {'foo': 'bar'}}]
+
+ input: {'a': [('b', 'number'), ('c', 'string', 'column c')]}
+ output: [{'id': 'a', 'label': 'a', 'type': 'string',
+ 'depth': 0, 'container': 'dict', 'custom_properties': {}},
+ {'id': 'b', 'label': 'b', 'type': 'number',
+ 'depth': 1, 'container': 'iter', 'custom_properties': {}},
+ {'id': 'c', 'label': 'column c', 'type': 'string',
+ 'depth': 1, 'container': 'iter', 'custom_properties': {}}]
+
+ input: {('a', 'number', 'column a'): { 'b': 'number', 'c': 'string'}}
+ output: [{'id': 'a', 'label': 'column a', 'type': 'number',
+ 'depth': 0, 'container': 'dict', 'custom_properties': {}},
+ {'id': 'b', 'label': 'b', 'type': 'number',
+ 'depth': 1, 'container': 'dict', 'custom_properties': {}},
+ {'id': 'c', 'label': 'c', 'type': 'string',
+ 'depth': 1, 'container': 'dict', 'custom_properties': {}}]
+
+ input: { ('w', 'string', 'word'): ('c', 'number', 'count') }
+ output: [{'id': 'w', 'label': 'word', 'type': 'string',
+ 'depth': 0, 'container': 'dict', 'custom_properties': {}},
+ {'id': 'c', 'label': 'count', 'type': 'number',
+ 'depth': 1, 'container': 'scalar', 'custom_properties': {}}]
+
+ input: {'a': ('number', 'column a'), 'b': ('string', 'column b')}
+ output: [{'id': 'a', 'label': 'column a', 'type': 'number', 'depth': 0,
+ 'container': 'dict', 'custom_properties': {}},
+ {'id': 'b', 'label': 'column b', 'type': 'string', 'depth': 0,
+ 'container': 'dict', 'custom_properties': {}}
+
+ NOTE: there might be ambiguity in the case of a dictionary representation
+ of a single column. For example, the following description can be parsed
+ in 2 different ways: {'a': ('b', 'c')} can be thought of a single column
+ with the id 'a', of type 'b' and the label 'c', or as 2 columns: one named
+ 'a', and the other named 'b' of type 'c'. We choose the first option by
+ default, and in case the second option is the right one, it is possible to
+ make the key into a tuple (i.e. {('a',): ('b', 'c')}) or add more info
+ into the tuple, thus making it look like this: {'a': ('b', 'c', 'b', {})}
+ -- second 'b' is the label, and {} is the custom properties field.
+ """
+ # For the recursion step, we check for a scalar object (string or tuple)
+ if isinstance(table_description, (types.StringTypes, tuple)):
+ parsed_col = DataTable.ColumnTypeParser(table_description)
+ parsed_col["depth"] = depth
+ parsed_col["container"] = "scalar"
+ return [parsed_col]
+
+ # Since it is not scalar, table_description must be iterable.
+ if not hasattr(table_description, "__iter__"):
+ raise DataTableException("Expected an iterable object, got %s" %
+ type(table_description))
+ if not isinstance(table_description, dict):
+ # We expects a non-dictionary iterable item.
+ columns = []
+ for desc in table_description:
+ parsed_col = DataTable.ColumnTypeParser(desc)
+ parsed_col["depth"] = depth
+ parsed_col["container"] = "iter"
+ columns.append(parsed_col)
+ if not columns:
+ raise DataTableException("Description iterable objects should not"
+ " be empty.")
+ return columns
+ # The other case is a dictionary
+ if not table_description:
+ raise DataTableException("Empty dictionaries are not allowed inside"
+ " description")
+
+ # To differentiate between the two cases of more levels below or this is
+ # the most inner dictionary, we consider the number of keys (more then one
+ # key is indication for most inner dictionary) and the type of the key and
+ # value in case of only 1 key (if the type of key is string and the type of
+ # the value is a tuple of 0-3 items, we assume this is the most inner
+ # dictionary).
+ # NOTE: this way of differentiating might create ambiguity. See docs.
+ if (len(table_description) != 1 or
+ (isinstance(table_description.keys()[0], types.StringTypes) and
+ isinstance(table_description.values()[0], tuple) and
+ len(table_description.values()[0]) < 4)):
+ # This is the most inner dictionary. Parsing types.
+ columns = []
+ # We sort the items, equivalent to sort the keys since they are unique
+ for key, value in sorted(table_description.items()):
+ # We parse the column type as (key, type) or (key, type, label) using
+ # ColumnTypeParser.
+ if isinstance(value, tuple):
+ parsed_col = DataTable.ColumnTypeParser((key,) + value)
+ else:
+ parsed_col = DataTable.ColumnTypeParser((key, value))
+ parsed_col["depth"] = depth
+ parsed_col["container"] = "dict"
+ columns.append(parsed_col)
+ return columns
+ # This is an outer dictionary, must have at most one key.
+ parsed_col = DataTable.ColumnTypeParser(table_description.keys()[0])
+ parsed_col["depth"] = depth
+ parsed_col["container"] = "dict"
+ return ([parsed_col] +
+ DataTable.TableDescriptionParser(table_description.values()[0],
+ depth=depth + 1))
+
+ @property
+ def columns(self):
+ """Returns the parsed table description."""
+ return self.__columns
+
+ def NumberOfRows(self):
+ """Returns the number of rows in the current data stored in the table."""
+ return len(self.__data)
+
+ def SetRowsCustomProperties(self, rows, custom_properties):
+ """Sets the custom properties for given row(s).
+
+ Can accept a single row or an iterable of rows.
+ Sets the given custom properties for all specified rows.
+
+ Args:
+ rows: The row, or rows, to set the custom properties for.
+ custom_properties: A string to string dictionary of custom properties to
+ set for all rows.
+ """
+ if not hasattr(rows, "__iter__"):
+ rows = [rows]
+ for row in rows:
+ self.__data[row] = (self.__data[row][0], custom_properties)
+
+ def LoadData(self, data, custom_properties=None):
+ """Loads new rows to the data table, clearing existing rows.
+
+ May also set the custom_properties for the added rows. The given custom
+ properties dictionary specifies the dictionary that will be used for *all*
+ given rows.
+
+ Args:
+ data: The rows that the table will contain.
+ custom_properties: A dictionary of string to string to set as the custom
+ properties for all rows.
+ """
+ self.__data = []
+ self.AppendData(data, custom_properties)
+
+ def AppendData(self, data, custom_properties=None):
+ """Appends new data to the table.
+
+ Data is appended in rows. Data must comply with
+ the table schema passed in to __init__(). See CoerceValue() for a list
+ of acceptable data types. See the class documentation for more information
+ and examples of schema and data values.
+
+ Args:
+ data: The row to add to the table. The data must conform to the table
+ description format.
+ custom_properties: A dictionary of string to string, representing the
+ custom properties to add to all the rows.
+
+ Raises:
+ DataTableException: The data structure does not match the description.
+ """
+ # If the maximal depth is 0, we simply iterate over the data table
+ # lines and insert them using _InnerAppendData. Otherwise, we simply
+ # let the _InnerAppendData handle all the levels.
+ if not self.__columns[-1]["depth"]:
+ for row in data:
+ self._InnerAppendData(({}, custom_properties), row, 0)
+ else:
+ self._InnerAppendData(({}, custom_properties), data, 0)
+
+ def _InnerAppendData(self, prev_col_values, data, col_index):
+ """Inner function to assist LoadData."""
+ # We first check that col_index has not exceeded the columns size
+ if col_index >= len(self.__columns):
+ raise DataTableException("The data does not match description, too deep")
+
+ # Dealing with the scalar case, the data is the last value.
+ if self.__columns[col_index]["container"] == "scalar":
+ prev_col_values[0][self.__columns[col_index]["id"]] = data
+ self.__data.append(prev_col_values)
+ return
+
+ if self.__columns[col_index]["container"] == "iter":
+ if not hasattr(data, "__iter__") or isinstance(data, dict):
+ raise DataTableException("Expected iterable object, got %s" %
+ type(data))
+ # We only need to insert the rest of the columns
+ # If there are less items than expected, we only add what there is.
+ for value in data:
+ if col_index >= len(self.__columns):
+ raise DataTableException("Too many elements given in data")
+ prev_col_values[0][self.__columns[col_index]["id"]] = value
+ col_index += 1
+ self.__data.append(prev_col_values)
+ return
+
+ # We know the current level is a dictionary, we verify the type.
+ if not isinstance(data, dict):
+ raise DataTableException("Expected dictionary at current level, got %s" %
+ type(data))
+ # We check if this is the last level
+ if self.__columns[col_index]["depth"] == self.__columns[-1]["depth"]:
+ # We need to add the keys in the dictionary as they are
+ for col in self.__columns[col_index:]:
+ if col["id"] in data:
+ prev_col_values[0][col["id"]] = data[col["id"]]
+ self.__data.append(prev_col_values)
+ return
+
+ # We have a dictionary in an inner depth level.
+ if not data.keys():
+ # In case this is an empty dictionary, we add a record with the columns
+ # filled only until this point.
+ self.__data.append(prev_col_values)
+ else:
+ for key in sorted(data):
+ col_values = dict(prev_col_values[0])
+ col_values[self.__columns[col_index]["id"]] = key
+ self._InnerAppendData((col_values, prev_col_values[1]),
+ data[key], col_index + 1)
+
+ def _PreparedData(self, order_by=()):
+ """Prepares the data for enumeration - sorting it by order_by.
+
+ Args:
+ order_by: Optional. Specifies the name of the column(s) to sort by, and
+ (optionally) which direction to sort in. Default sort direction
+ is asc. Following formats are accepted:
+ "string_col_name" -- For a single key in default (asc) order.
+ ("string_col_name", "asc|desc") -- For a single key.
+ [("col_1","asc|desc"), ("col_2","asc|desc")] -- For more than
+ one column, an array of tuples of (col_name, "asc|desc").
+
+ Returns:
+ The data sorted by the keys given.
+
+ Raises:
+ DataTableException: Sort direction not in 'asc' or 'desc'
+ """
+ if not order_by:
+ return self.__data
+
+ proper_sort_keys = []
+ if isinstance(order_by, types.StringTypes) or (
+ isinstance(order_by, tuple) and len(order_by) == 2 and
+ order_by[1].lower() in ["asc", "desc"]):
+ order_by = (order_by,)
+ for key in order_by:
+ if isinstance(key, types.StringTypes):
+ proper_sort_keys.append((key, 1))
+ elif (isinstance(key, (list, tuple)) and len(key) == 2 and
+ key[1].lower() in ("asc", "desc")):
+ proper_sort_keys.append((key[0], key[1].lower() == "asc" and 1 or -1))
+ else:
+ raise DataTableException("Expected tuple with second value: "
+ "'asc' or 'desc'")
+
+ def SortCmpFunc(row1, row2):
+ """cmp function for sorted. Compares by keys and 'asc'/'desc' keywords."""
+ for key, asc_mult in proper_sort_keys:
+ cmp_result = asc_mult * cmp(row1[0].get(key), row2[0].get(key))
+ if cmp_result:
+ return cmp_result
+ return 0
+
+ return sorted(self.__data, cmp=SortCmpFunc)
+
+ def ToJSCode(self, name, columns_order=None, order_by=()):
+ """Writes the data table as a JS code string.
+
+ This method writes a string of JS code that can be run to
+ generate a DataTable with the specified data. Typically used for debugging
+ only.
+
+ Args:
+ name: The name of the table. The name would be used as the DataTable's
+ variable name in the created JS code.
+ columns_order: Optional. Specifies the order of columns in the
+ output table. Specify a list of all column IDs in the order
+ in which you want the table created.
+ Note that you must list all column IDs in this parameter,
+ if you use it.
+ order_by: Optional. Specifies the name of the column(s) to sort by.
+ Passed as is to _PreparedData.
+
+ Returns:
+ A string of JS code that, when run, generates a DataTable with the given
+ name and the data stored in the DataTable object.
+ Example result:
+ "var tab1 = new google.visualization.DataTable();
+ tab1.addColumn("string", "a", "a");
+ tab1.addColumn("number", "b", "b");
+ tab1.addColumn("boolean", "c", "c");
+ tab1.addRows(10);
+ tab1.setCell(0, 0, "a");
+ tab1.setCell(0, 1, 1, null, {"foo": "bar"});
+ tab1.setCell(0, 2, true);
+ ...
+ tab1.setCell(9, 0, "c");
+ tab1.setCell(9, 1, 3, "3$");
+ tab1.setCell(9, 2, false);"
+
+ Raises:
+ DataTableException: The data does not match the type.
+ """
+
+ encoder = DataTableJSONEncoder()
+
+ if columns_order is None:
+ columns_order = [col["id"] for col in self.__columns]
+ col_dict = dict([(col["id"], col) for col in self.__columns])
+
+ # We first create the table with the given name
+ jscode = "var %s = new google.visualization.DataTable();\n" % name
+ if self.custom_properties:
+ jscode += "%s.setTableProperties(%s);\n" % (
+ name, encoder.encode(self.custom_properties))
+
+ # We add the columns to the table
+ for i, col in enumerate(columns_order):
+ jscode += "%s.addColumn(%s, %s, %s);\n" % (
+ name,
+ encoder.encode(col_dict[col]["type"]),
+ encoder.encode(col_dict[col]["label"]),
+ encoder.encode(col_dict[col]["id"]))
+ if col_dict[col]["custom_properties"]:
+ jscode += "%s.setColumnProperties(%d, %s);\n" % (
+ name, i, encoder.encode(col_dict[col]["custom_properties"]))
+ jscode += "%s.addRows(%d);\n" % (name, len(self.__data))
+
+ # We now go over the data and add each row
+ for (i, (row, cp)) in enumerate(self._PreparedData(order_by)):
+ # We add all the elements of this row by their order
+ for (j, col) in enumerate(columns_order):
+ if col not in row or row[col] is None:
+ continue
+ value = self.CoerceValue(row[col], col_dict[col]["type"])
+ if isinstance(value, tuple):
+ cell_cp = ""
+ if len(value) == 3:
+ cell_cp = ", %s" % encoder.encode(row[col][2])
+ # We have a formatted value or custom property as well
+ jscode += ("%s.setCell(%d, %d, %s, %s%s);\n" %
+ (name, i, j,
+ self.EscapeForJSCode(encoder, value[0]),
+ self.EscapeForJSCode(encoder, value[1]), cell_cp))
+ else:
+ jscode += "%s.setCell(%d, %d, %s);\n" % (
+ name, i, j, self.EscapeForJSCode(encoder, value))
+ if cp:
+ jscode += "%s.setRowProperties(%d, %s);\n" % (
+ name, i, encoder.encode(cp))
+ return jscode
+
+ def ToHtml(self, columns_order=None, order_by=()):
+ """Writes the data table as an HTML table code string.
+
+ Args:
+ columns_order: Optional. Specifies the order of columns in the
+ output table. Specify a list of all column IDs in the order
+ in which you want the table created.
+ Note that you must list all column IDs in this parameter,
+ if you use it.
+ order_by: Optional. Specifies the name of the column(s) to sort by.
+ Passed as is to _PreparedData.
+
+ Returns:
+ An HTML table code string.
+ Example result (the result is without the newlines):
+ <html><body><table border="1">
+ <thead><tr><th>a</th><th>b</th><th>c</th></tr></thead>
+ <tbody>
+ <tr><td>1</td><td>"z"</td><td>2</td></tr>
+ <tr><td>"3$"</td><td>"w"</td><td></td></tr>
+ </tbody>
+ </table></body></html>
+
+ Raises:
+ DataTableException: The data does not match the type.
+ """
+ table_template = "<html><body><table border=\"1\">%s</table></body></html>"
+ columns_template = "<thead><tr>%s</tr></thead>"
+ rows_template = "<tbody>%s</tbody>"
+ row_template = "<tr>%s</tr>"
+ header_cell_template = "<th>%s</th>"
+ cell_template = "<td>%s</td>"
+
+ if columns_order is None:
+ columns_order = [col["id"] for col in self.__columns]
+ col_dict = dict([(col["id"], col) for col in self.__columns])
+
+ columns_list = []
+ for col in columns_order:
+ columns_list.append(header_cell_template %
+ cgi.escape(col_dict[col]["label"]))
+ columns_html = columns_template % "".join(columns_list)
+
+ rows_list = []
+ # We now go over the data and add each row
+ for row, unused_cp in self._PreparedData(order_by):
+ cells_list = []
+ # We add all the elements of this row by their order
+ for col in columns_order:
+ # For empty string we want empty quotes ("").
+ value = ""
+ if col in row and row[col] is not None:
+ value = self.CoerceValue(row[col], col_dict[col]["type"])
+ if isinstance(value, tuple):
+ # We have a formatted value and we're going to use it
+ cells_list.append(cell_template % cgi.escape(self.ToString(value[1])))
+ else:
+ cells_list.append(cell_template % cgi.escape(self.ToString(value)))
+ rows_list.append(row_template % "".join(cells_list))
+ rows_html = rows_template % "".join(rows_list)
+
+ return table_template % (columns_html + rows_html)
+
+ def ToCsv(self, columns_order=None, order_by=(), separator=","):
+ """Writes the data table as a CSV string.
+
+ Output is encoded in UTF-8 because the Python "csv" module can't handle
+ Unicode properly according to its documentation.
+
+ Args:
+ columns_order: Optional. Specifies the order of columns in the
+ output table. Specify a list of all column IDs in the order
+ in which you want the table created.
+ Note that you must list all column IDs in this parameter,
+ if you use it.
+ order_by: Optional. Specifies the name of the column(s) to sort by.
+ Passed as is to _PreparedData.
+ separator: Optional. The separator to use between the values.
+
+ Returns:
+ A CSV string representing the table.
+ Example result:
+ 'a','b','c'
+ 1,'z',2
+ 3,'w',''
+
+ Raises:
+ DataTableException: The data does not match the type.
+ """
+
+ csv_buffer = cStringIO.StringIO()
+ writer = csv.writer(csv_buffer, delimiter=separator)
+
+ if columns_order is None:
+ columns_order = [col["id"] for col in self.__columns]
+ col_dict = dict([(col["id"], col) for col in self.__columns])
+
+ writer.writerow([col_dict[col]["label"].encode("utf-8")
+ for col in columns_order])
+
+ # We now go over the data and add each row
+ for row, unused_cp in self._PreparedData(order_by):
+ cells_list = []
+ # We add all the elements of this row by their order
+ for col in columns_order:
+ value = ""
+ if col in row and row[col] is not None:
+ value = self.CoerceValue(row[col], col_dict[col]["type"])
+ if isinstance(value, tuple):
+ # We have a formatted value. Using it only for date/time types.
+ if col_dict[col]["type"] in ["date", "datetime", "timeofday"]:
+ cells_list.append(self.ToString(value[1]).encode("utf-8"))
+ else:
+ cells_list.append(self.ToString(value[0]).encode("utf-8"))
+ else:
+ cells_list.append(self.ToString(value).encode("utf-8"))
+ writer.writerow(cells_list)
+ return csv_buffer.getvalue()
+
+ def ToTsvExcel(self, columns_order=None, order_by=()):
+ """Returns a file in tab-separated-format readable by MS Excel.
+
+ Returns a file in UTF-16 little endian encoding, with tabs separating the
+ values.
+
+ Args:
+ columns_order: Delegated to ToCsv.
+ order_by: Delegated to ToCsv.
+
+ Returns:
+ A tab-separated little endian UTF16 file representing the table.
+ """
+ return (self.ToCsv(columns_order, order_by, separator="\t")
+ .decode("utf-8").encode("UTF-16LE"))
+
+ def _ToJSonObj(self, columns_order=None, order_by=()):
+ """Returns an object suitable to be converted to JSON.
+
+ Args:
+ columns_order: Optional. A list of all column IDs in the order in which
+ you want them created in the output table. If specified,
+ all column IDs must be present.
+ order_by: Optional. Specifies the name of the column(s) to sort by.
+ Passed as is to _PreparedData().
+
+ Returns:
+ A dictionary object for use by ToJSon or ToJSonResponse.
+ """
+ if columns_order is None:
+ columns_order = [col["id"] for col in self.__columns]
+ col_dict = dict([(col["id"], col) for col in self.__columns])
+
+ # Creating the column JSON objects
+ col_objs = []
+ for col_id in columns_order:
+ col_obj = {"id": col_dict[col_id]["id"],
+ "label": col_dict[col_id]["label"],
+ "type": col_dict[col_id]["type"]}
+ if col_dict[col_id]["custom_properties"]:
+ col_obj["p"] = col_dict[col_id]["custom_properties"]
+ col_objs.append(col_obj)
+
+ # Creating the rows jsons
+ row_objs = []
+ for row, cp in self._PreparedData(order_by):
+ cell_objs = []
+ for col in columns_order:
+ value = self.CoerceValue(row.get(col, None), col_dict[col]["type"])
+ if value is None:
+ cell_obj = None
+ elif isinstance(value, tuple):
+ cell_obj = {"v": value[0]}
+ if len(value) > 1 and value[1] is not None:
+ cell_obj["f"] = value[1]
+ if len(value) == 3:
+ cell_obj["p"] = value[2]
+ else:
+ cell_obj = {"v": value}
+ cell_objs.append(cell_obj)
+ row_obj = {"c": cell_objs}
+ if cp:
+ row_obj["p"] = cp
+ row_objs.append(row_obj)
+
+ json_obj = {"cols": col_objs, "rows": row_objs}
+ if self.custom_properties:
+ json_obj["p"] = self.custom_properties
+
+ return json_obj
+
+ def ToJSon(self, columns_order=None, order_by=()):
+ """Returns a string that can be used in a JS DataTable constructor.
+
+ This method writes a JSON string that can be passed directly into a Google
+ Visualization API DataTable constructor. Use this output if you are
+ hosting the visualization HTML on your site, and want to code the data
+ table in Python. Pass this string into the
+ google.visualization.DataTable constructor, e.g,:
+ ... on my page that hosts my visualization ...
+ google.setOnLoadCallback(drawTable);
+ function drawTable() {
+ var data = new google.visualization.DataTable(_my_JSon_string, 0.6);
+ myTable.draw(data);
+ }
+
+ Args:
+ columns_order: Optional. Specifies the order of columns in the
+ output table. Specify a list of all column IDs in the order
+ in which you want the table created.
+ Note that you must list all column IDs in this parameter,
+ if you use it.
+ order_by: Optional. Specifies the name of the column(s) to sort by.
+ Passed as is to _PreparedData().
+
+ Returns:
+ A JSon constructor string to generate a JS DataTable with the data
+ stored in the DataTable object.
+ Example result (the result is without the newlines):
+ {cols: [{id:"a",label:"a",type:"number"},
+ {id:"b",label:"b",type:"string"},
+ {id:"c",label:"c",type:"number"}],
+ rows: [{c:[{v:1},{v:"z"},{v:2}]}, c:{[{v:3,f:"3$"},{v:"w"},null]}],
+ p: {'foo': 'bar'}}
+
+ Raises:
+ DataTableException: The data does not match the type.
+ """
+
+ encoder = DataTableJSONEncoder()
+ return encoder.encode(
+ self._ToJSonObj(columns_order, order_by)).encode("utf-8")
+
+ def ToJSonResponse(self, columns_order=None, order_by=(), req_id=0,
+ response_handler="google.visualization.Query.setResponse"):
+ """Writes a table as a JSON response that can be returned as-is to a client.
+
+ This method writes a JSON response to return to a client in response to a
+ Google Visualization API query. This string can be processed by the calling
+ page, and is used to deliver a data table to a visualization hosted on
+ a different page.
+
+ Args:
+ columns_order: Optional. Passed straight to self.ToJSon().
+ order_by: Optional. Passed straight to self.ToJSon().
+ req_id: Optional. The response id, as retrieved by the request.
+ response_handler: Optional. The response handler, as retrieved by the
+ request.
+
+ Returns:
+ A JSON response string to be received by JS the visualization Query
+ object. This response would be translated into a DataTable on the
+ client side.
+ Example result (newlines added for readability):
+ google.visualization.Query.setResponse({
+ 'version':'0.6', 'reqId':'0', 'status':'OK',
+ 'table': {cols: [...], rows: [...]}});
+
+ Note: The URL returning this string can be used as a data source by Google
+ Visualization Gadgets or from JS code.
+ """
+
+ response_obj = {
+ "version": "0.6",
+ "reqId": str(req_id),
+ "table": self._ToJSonObj(columns_order, order_by),
+ "status": "ok"
+ }
+ encoder = DataTableJSONEncoder()
+ return "%s(%s);" % (response_handler,
+ encoder.encode(response_obj).encode("utf-8"))
+
+ def ToResponse(self, columns_order=None, order_by=(), tqx=""):
+ """Writes the right response according to the request string passed in tqx.
+
+ This method parses the tqx request string (format of which is defined in
+ the documentation for implementing a data source of Google Visualization),
+ and returns the right response according to the request.
+ It parses out the "out" parameter of tqx, calls the relevant response
+ (ToJSonResponse() for "json", ToCsv() for "csv", ToHtml() for "html",
+ ToTsvExcel() for "tsv-excel") and passes the response function the rest of
+ the relevant request keys.
+
+ Args:
+ columns_order: Optional. Passed as is to the relevant response function.
+ order_by: Optional. Passed as is to the relevant response function.
+ tqx: Optional. The request string as received by HTTP GET. Should be in
+ the format "key1:value1;key2:value2...". All keys have a default
+ value, so an empty string will just do the default (which is calling
+ ToJSonResponse() with no extra parameters).
+
+ Returns:
+ A response string, as returned by the relevant response function.
+
+ Raises:
+ DataTableException: One of the parameters passed in tqx is not supported.
+ """
+ tqx_dict = {}
+ if tqx:
+ tqx_dict = dict(opt.split(":") for opt in tqx.split(";"))
+ if tqx_dict.get("version", "0.6") != "0.6":
+ raise DataTableException(
+ "Version (%s) passed by request is not supported."
+ % tqx_dict["version"])
+
+ if tqx_dict.get("out", "json") == "json":
+ response_handler = tqx_dict.get("responseHandler",
+ "google.visualization.Query.setResponse")
+ return self.ToJSonResponse(columns_order, order_by,
+ req_id=tqx_dict.get("reqId", 0),
+ response_handler=response_handler)
+ elif tqx_dict["out"] == "html":
+ return self.ToHtml(columns_order, order_by)
+ elif tqx_dict["out"] == "csv":
+ return self.ToCsv(columns_order, order_by)
+ elif tqx_dict["out"] == "tsv-excel":
+ return self.ToTsvExcel(columns_order, order_by)
+ else:
+ raise DataTableException(
+ "'out' parameter: '%s' is not supported" % tqx_dict["out"])
diff --git a/tools/demo/demo_reporter.py b/tools/demo/demo_reporter.py
new file mode 100755
index 0000000..d79fdac
--- /dev/null
+++ b/tools/demo/demo_reporter.py
@@ -0,0 +1,81 @@
+#!/usr/bin/env python
+# Copyright 2017 The Fuchsia Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+""" This script is used as part of the Cobalt demo. It provides a command-line
+ interface that allows an operator to request the generation of the two
+ example reports. Report 1 is for the Forculus / URL example. The report
+ is presented as a table of numbers. Report 2 is the Basic RAPPOR / hour-of-
+ the-day example. In addition to generating a table of numbers a
+ column-chart is also generated for visualization.
+"""
+
+import os
+import subprocess
+import sys
+
+import generate_viz
+
+DEMO_DIR = os.path.abspath(os.path.dirname(__file__))
+ROOT_DIR = os.path.abspath(os.path.join(DEMO_DIR, os.path.pardir, os.path.pardir))
+OUT_DIR = os.path.join(ROOT_DIR,'out')
+
+def runForculusDemo():
+ path =os.path.join(OUT_DIR, 'tools', 'report_client')
+ return_code =subprocess.call([path,
+ "-report_master_uri", "localhost:7001",
+ "-interactive=false",
+ "-report_config_id=1",
+ "-include_std_err_column=false",
+ "-logtostderr", "-v=3"])
+ if return_code < 0:
+ print
+ print "****** WARNING Process terminated by signal %d" % (- return_code)
+
+def runBasicRapporDemo():
+ path =os.path.join(OUT_DIR, 'tools', 'report_client')
+ return_code =subprocess.call([path,
+ "-report_master_uri", "localhost:7001",
+ "-interactive=false",
+ "-report_config_id=2",
+ "-include_std_err_column=true",
+ "-csv_file=%s" % generate_viz.USAGE_BY_HOUR_CSV_FILE,
+ "-logtostderr", "-v=3"])
+ if return_code < 0:
+ print
+ print "****** WARNING Process terminated by signal %d" % (- return_code)
+ # Gemerate the vizualization
+ generate_viz.generateViz()
+
+def main():
+ while True:
+ print "Cobalt Demo"
+ print "----------"
+ print "1) Run Forculus report demo"
+ print
+ print "2) Run Basic RAPPOR report demo"
+ print
+ print "3) Quit"
+ print
+ print "Enter 1, 2 or 3"
+ line = sys.stdin.readline()
+ if line == '1\n':
+ runForculusDemo()
+ elif line == '2\n':
+ runBasicRapporDemo()
+ elif line == '3\n':
+ break
+
+if __name__ == '__main__':
+ main()
diff --git a/tools/demo/generate_viz.py b/tools/demo/generate_viz.py
new file mode 100755
index 0000000..4970160
--- /dev/null
+++ b/tools/demo/generate_viz.py
@@ -0,0 +1,129 @@
+#!/usr/bin/env python
+# Copyright 2017 The Fuchsia Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+""" This script is used as part of the Cobalt demo in order to generate a
+ chart for the Basic RAPPOR / hour-of-the-day example. The script reads a
+ csv files containing data to be visualized and uses the Google Data
+ Visualization API to transform the data into a JavaScript DataTable.
+ The generated JavaScript is then imported by visualization.html and used
+ to generate a chart.
+"""
+
+import csv
+import os
+import sys
+
+DEMO_DIR = os.path.abspath(os.path.dirname(__file__))
+ROOT_DIR = os.path.abspath(
+ os.path.join(DEMO_DIR, os.path.pardir, os.path.pardir))
+
+sys.path.insert(0, ROOT_DIR)
+import third_party.google_visualization.gviz_api as gviz_api
+
+OUT_DIR = os.path.join(ROOT_DIR,'out')
+
+# The output JavaScript file to be created.
+OUTPUT_JS_FILE = os.path.join(OUT_DIR, 'demo_viz_data.js')
+
+# The input csv file to be read.
+USAGE_BY_HOUR_CSV_FILE = os.path.join(OUT_DIR, 'usage_by_hour.csv')
+
+# The HTML file
+VIZ_HTML_FILE = os.path.join(DEMO_DIR, 'visualization.html')
+
+# The javascript variables to write.
+USAGE_BY_HOUR_JS_VAR_NAME = 'usage_by_hour_data'
+
+def buildDataTableJs(data=None, var_name=None, description=None,
+ columns_order=None, order_by=()):
+ """Builds a JavaScript string defining a DataTable containing the given data.
+
+ Args:
+ data: {dictionary}: The data with which to populate the DataTable.
+ var_name {string}: The name of the JavaScript variable to write.
+ description {dictionary}: Passed to the constructor of gviz_api.DataTable()
+ columns_order {tuple of string}: The names of the table columns in the
+ order they should be written. Optional.
+ order_by {tuple of string}: Optional. Specify something like ('foo', 'des')
+ to sort the rows by the 'foo' column in descending order.
+
+ Returns:
+ {string} of the form |var_name|=<json>, where <json> is a json string
+ defining a data table.
+ """
+ # Load data into a gviz_api.DataTable
+ data_table = gviz_api.DataTable(description)
+ data_table.LoadData(data)
+ json = data_table.ToJSon(columns_order=columns_order,order_by=order_by)
+
+ return "%s=%s;" % (var_name, json)
+
+def buildUsageByHourJs():
+ """Builds a string defining variables used for visualization.
+
+ Reads the CSV file containing the usage-by-hour data and builds a JavaScript
+ string defining a DataTable containing the data
+
+ Returns: {string} of the form <var_name>=<json>, where |json| is a json string
+ defining a data table abd |var_name|s is USAGE_BY_HOUR_JS_VAR_NAME.
+ """
+ # The CSV file is the output of the RAPPOR analyzer.
+ # We read it and put the data into a dictionary.
+ # We are going to visualize the data as an interval chart and so we want to
+ # compute the high and low 95% confidence interval values wich we may do
+ # using the "std_error" column, column 2.
+ if (not os.path.exists(USAGE_BY_HOUR_CSV_FILE)):
+ print "File not found: %s" % USAGE_BY_HOUR_CSV_FILE
+ return None
+ with open(USAGE_BY_HOUR_CSV_FILE, 'rb') as csvfile:
+ reader = csv.reader(csvfile)
+ data = [{"hour" : int(row[0]), "estimate": max(float(row[1]), 0),
+ "low" : max(float(row[1]) - 1.96 * float(row[2]), 0),
+ "high": float(row[1]) + 1.96 * float(row[2])}
+ for row in reader if reader.line_num > 1]
+ usage_by_hour_cobalt_js = buildDataTableJs(
+ data=data,
+ var_name=USAGE_BY_HOUR_JS_VAR_NAME,
+ description={"hour": ("number", "Hour"),
+ "estimate": ("number", "Estimate"),
+ # The role: 'interval' property is what tells the Google
+ # Visualization API to draw an interval chart.
+ "low": ("number", "Low", {'role': 'interval'}),
+ "high": ("number", "High", {'role': 'interval'})},
+ columns_order=("hour", "estimate", "low", "high"),
+ order_by=("hour", "asc"))
+
+ return usage_by_hour_cobalt_js
+
+def generateViz():
+ print "Generating visualization..."
+
+ # Read the input files and build the JavaScript strings to write.
+ usage_by_hour_js = buildUsageByHourJs()
+ if usage_by_hour_js is None:
+ return
+
+ # Write the output file.
+ with open(OUTPUT_JS_FILE, 'w+b') as f:
+ f.write("// This js file is generated by the script "
+ "generate_viz.py\n\n")
+
+ f.write("%s\n\n" % usage_by_hour_js)
+
+ f.write("")
+
+ print "View the vizualization in your browser:"
+ print "file://%s" % VIZ_HTML_FILE
+
diff --git a/tools/demo/visualization.html b/tools/demo/visualization.html
new file mode 100644
index 0000000..abbc50b
--- /dev/null
+++ b/tools/demo/visualization.html
@@ -0,0 +1,95 @@
+<html>
+ <head>
+ <!--Load the AJAX API-->
+ <script type="text/javascript" src="https://www.gstatic.com/charts/loader.js"></script>
+ <script type="text/javascript" src="https://www.google.com/jsapi"></script>
+ <!-- Defines the variables named foo_data -->
+ <script type="text/javascript" src="../../out/demo_viz_data.js"></script>
+
+ <script type="text/javascript">
+
+ // Load the Visualization API and some packages.
+ google.charts.load('current', {'packages':['corechart', 'geochart', 'table']});
+
+ // Set a callback to run when the Google Visualization API is loaded.
+ google.charts.setOnLoadCallback(drawCharts);
+
+
+ // Callback that invokes the chart-drawing functions.
+ // NOTE: "_SC" refers to charts that represent data from the
+ // "Straight Counting" pipeline.
+ function drawCharts() {
+ drawUsageByHourColumnChart();
+ };
+
+ function drawUsageByHourColumnChart() {
+ var title = 'Fuchsia usage by hour, estimates with 95% Confidence Intervals';
+
+ // Parse the data table.
+ var data = new google.visualization.DataTable(usage_by_hour_data);
+ var options = {
+ title: title,
+ 'width': 900,
+ 'height': 400,
+ intervals: { color: 'black', 'lineWidth':2, 'barWidth': 1.2},
+ hAxis: {
+ title: 'Hour of Day',
+ gridlines: {count: 12},
+ },
+ vAxis: {viewWindow: {max: 1200}},
+ };
+
+ // Instantiate and draw our chart.
+ var chart = new google.visualization.ColumnChart(document.getElementById('usage_by_hour_chart'));
+ chart.draw(data, options);
+ }
+
+ function drawTable(data_var, element_id) {
+ // Parse the data table.
+ var data = new google.visualization.DataTable(data_var);
+
+ // Instantiate and draw our table.
+ var table = new google.visualization.Table(document.getElementById(element_id));
+
+ table.draw(data, {
+ showRowNumber: false,
+ alternatingRowStyle: true,
+ sortAscending: false,
+ allowHtml: true,
+ cssClassNames: {headerRow: 'table_header'},
+ width: 500,
+ frozenColumns: 1,
+ });
+ }
+
+ function drawPopularUrlsTables() {
+ drawTable(popular_urls_data_sc, 'popular_urls_table_sc')
+ drawTable(popular_urls_data, 'popular_urls_table')
+ }
+
+ function drawPopularHelpQueriesTables() {
+ drawTable(popular_help_queries_data_sc, 'popular_help_queries_table_sc')
+ drawTable(popular_help_queries_data, 'popular_help_queries_table')
+ }
+
+ </script>
+ </head>
+
+ <style>
+.table_header {
+ text-align: left;
+ }
+ </style>
+
+ <body>
+ <table>
+ <tr>
+ <td>
+ <div id="usage_by_hour_chart"></div>
+ </td>
+ </tr>
+ </table>
+
+
+ </body>
+</html>
