963f774818
* To find the oldest and newest quantities for calculating rate, Spark Data Frame is converted to a Spark RDD, which makes it easy to do a group by and sort. The syntax to pull the value from a map column type in an RDD is rdd.map_column['key'] which is different from dataframe.map_column.key which was causing 'dict' object has no attribute exception * Renamed first_attempt_at_spark_test.py to test_first_attempt_at_spark, because all the tests in that file were not being run. * Changed the pre transform and transform specs which were being used by test_first_attempt_at_spark to newer 'dimensions#field_name that was introduced in rocky release. Story: 2005449 Task: 30501 Change-Id: I6adeee54fe261c535372b8f5f3580e7d3261259b |
||
---|---|---|
.. | ||
builder | ||
grouping | ||
__init__.py | ||
storage_utils.py | ||
transform_utils.py |