Parquet reporting columns

Modified on Tue, 25 Aug at 1:17 PM

Every column of the ten reporting files in a Parquet snapshot — the derived, denormalised reporting view — with its physical type, whether it can be empty, and what it holds. It is a lookup table for data engineers and report developers who are already reading the export and need to know what a column means; for how snapshots are laid out and how to tell a finished one from a partial write, see Parquet export.

Contents

How to read these tables

Each section lists the fixed columns of one file: the columns every snapshot of that file has, on every deployment. Metadata columns are extra, differ per customer, and are covered separately below.

Every column is physically nullable. The writer emits an optional Parquet field for all of them, so the file's schema marks nothing as required. The Empty? column therefore describes the data rather than the schema:

  • never — the value is a non-optional part of the entity and every row carries one.
  • possible — the value is genuinely optional, and a consumer has to handle null.

A consumer that tries to read a non-null constraint out of the schema will not find one.

Column order is not a contract. Columns are discovered by reflection over the domain model, most-derived declarations first and inherited ones last — which is why Id and Created sit at the end of almost every table rather than the start. These tables list them in the order the file actually stores them, but that order can shift when the model changes. Address columns by name.

Enum columns carry names, not numbers. Where a column is an enum its permitted values are listed under the table. This is worth checking if you are moving a query over from the REST export, which serves the same values as integers alongside a separate …Text field: compare a Parquet enum column against a number and you get an empty result with no error to tell you why.

Revisions are columns, and the two big files name them differently. Both denormalise the revision comparison into columns, so comparing Live against a baseline at one cut-off is a comparison between columns of one file — no join, no filter. But the naming is not consistent between them:

Live Baseline Original baseline Current Revised
reportActivity.parquet suffixes Start StartBaseline StartBaselineOriginal StartCurrent StartRevised
reportActivityPeriodized.parquet prefixes PlannedHours BaselinePlannedHours OriginalBaselinePlannedHours CurrentPlannedHours RevisedPlannedHours

In both, the bare form is Live. Authoring a measure against the wrong convention is the most common way to get a plausible number out of the wrong column.

BaseLine is capitalised inconsistently, and it is not a typo in this page. Most reporting files carry a column pointing back at the planning row of each revision, and the baseline one is spelled with a capital L in five files and a lowercase l in three:

Spelling Files
BaseLine…Id reportResource, reportCalendar, reportProfile, reportResourceAssignment, reportSuccessor
Baseline…Id reportActivity, reportStructure, reportStructureElement

OriginalBaseline…Id is lowercase l everywhere. Code that builds a column name by concatenating a revision name onto an entity name will silently miss half of these, so read the spelling off the table in each section rather than deriving it.

Obsolete columns

Progress is only ever reported against Live scope. An actual — earned hours, expended hours, current progress — attached to any other revision therefore carries no meaning, even though the column is still written and still holds a number. Eight columns are affected, and each is marked Obsolete. where it appears below:

File Obsolete Use instead
reportActivity.parquet EarnedWorkHoursAtCutoffPeriodicBaseline EarnedWorkHoursAtCutoffPeriodic
reportActivity.parquet EarnedWorkHoursAtCutoffCumulativeBaseline EarnedWorkHoursAtCutoffCumulative
reportActivity.parquet CurrentProgressBaseline CurrentProgress
reportActivityPeriodized.parquet BaselineEarnedHours EarnedHours
reportActivityPeriodized.parquet OriginalBaselineTotalMinusActualHours TotalMinusActualHours
reportActivityPeriodized.parquet BaselineTotalMinusActualHours TotalMinusActualHours
reportActivityPeriodized.parquet CurrentTotalMinusActualHours TotalMinusActualHours
reportActivityPeriodized.parquet RevisedTotalMinusActualHours TotalMinusActualHours

The rule is about actuals, not about the word "earned". It reaches a column two ways — the column may be an actual, as the earned-hours and progress ones are, or it may be derived from one. The four TotalMinusActualHours variants are the derived case: each subtracts hours expended from that revision's planned total, but the hours expended are always Live, so a non-Live variant silently mixes one revision's estimate with another's actuals. The bare TotalMinusActualHours is Live on both sides and is the only one that holds together.

Two edges worth knowing, so the rule is not over-applied:

  • Estimates keep every revision variant. PlannedProgressBaseline and TotalFloatHoursBaseline sit immediately beside CurrentProgressBaseline and are perfectly valid — planned progress and float are estimates, and estimates are comparable across all five revisions. Dates and planned hours likewise, including the TotalPlannedHours, PlannedHours, PlannedLoad, CalendarHours and RemainingHours families that sit beside the retired TotalMinusActualHours variants.
  • ActualWorkHoursAtCutoffPeriodic and ActualWorkHoursAtCutoffCumulative have no non-Live variants at all, so there is nothing there to avoid.

Metadata columns

Alongside the fixed columns above, a file can carry one column per metadata field. Only one reporting file does: reportActivity.parquet. Every other reporting file has fixed columns only, and each section below says so.

The asymmetry with the planning dataset is deliberate, not an oversight: resource.parquet and resourceAssignment.parquet carry metadata columns, but reportResource.parquet and reportResourceAssignment.parquet carry none. If you are looking for a resource metadata field in the reporting dataset, it is in the planning files.

The set is not driven by your schedule type configuration. It is driven by the values that are actually there: the export collects the distinct metadata fields that have a value on the rows in scope, so a field configured for your deployment but never populated produces no column at all. Three consequences worth planning for:

  • The column set differs between customers, between two schedules of one customer, and between two cut-offs of the same schedule. A field populated for the first time appears as a new column in the next snapshot only, and earlier snapshots are not rewritten to gain it.
  • The scope is the report schedule and the revision being exported — reportActivity.parquet carries the fields with values on the report activities of this report schedule, for that revision. A non-active revision is exported planning-only, so it has no reporting file to carry them at all.
  • Numeric types collapse. A metadata field's numeric value is stored in a single numeric column whatever its declared type, so integer, decimal and floating-point fields all arrive as double. Boolean fields arrive as boolean, date fields as timestamp, and everything else as string.

A metadata column's name is not its identity. Names are sanitised and de-duplicated against the names already taken, so a metadata field can arrive under a suffixed name — a field called Description becomes Description_2, because the fixed Description column claimed the name first. No two fixed columns collide anywhere in the export, so a _2 or _3 suffix is always a metadata field that lost a name race, either against a fixed column or against a metadata field resolved before it. The suffixes are handed out in order, so adding a field upstream can move an existing column from _2 to _3.

Resolve metadata columns through _manifest.json, which maps each physical column back to its IlapId, before combining snapshots or comparing deployments. Parquet export covers the manifest in full.

reportActivity.parquet

One row per activity, with the dates, work-hour totals and progress, and each reference revision carried as its own column. Date drift is measured here and nowhere else.

Column Type Empty? Meaning
Description string possible Free-text description of the activity.
IsCancelled boolean possible Whether the activity has been cancelled.
CancelledDate timestamp possible When the activity was cancelled. Empty unless it was.
IsDeleted boolean never Whether the activity is marked deleted in the reporting view.
DeletedDate timestamp possible When the activity was marked deleted. Empty unless it was.
ActualFinish timestamp possible When the activity actually finished. Empty until it has.
ActualStart timestamp possible When the activity actually started. Empty until it has.
Updated timestamp possible When the reporting row was last updated.
Start timestamp possible Live start date.
Finish timestamp possible Live finish date.
StartBaseline timestamp possible Start date in the baseline revision.
FinishBaseline timestamp possible Finish date in the baseline revision.
StartBaselineOriginal timestamp possible Start date in the original baseline revision.
FinishBaselineOriginal timestamp possible Finish date in the original baseline revision.
StartCurrent timestamp possible Start date in the current revision.
FinishCurrent timestamp possible Finish date in the current revision.
StartRevised timestamp possible Start date in the revised revision.
FinishRevised timestamp possible Finish date in the revised revision.
TotalWorkHours double never Total work hours for the activity, Live.
TotalWorkHoursBaseline double never Total work hours in the baseline revision.
TotalWorkHoursBaselineOriginal double never Total work hours in the original baseline revision.
TotalWorkHoursCurrent double never Total work hours in the current revision.
TotalWorkHoursRevised double never Total work hours in the revised revision.
PlannedWorkHoursAtCutoffPeriodic double never Work hours planned within the cut-off period, Live.
PlannedWorkHoursAtCutoffCumulative double never Work hours planned up to the cut-off, Live.
EarnedWorkHoursAtCutoffPeriodic double never Work hours earned within the cut-off period.
EarnedWorkHoursAtCutoffPeriodicBaseline double never Obsolete. Earned hours against the baseline for the cut-off period. Progress is only reported against Live, so this carries no meaning; use EarnedWorkHoursAtCutoffPeriodic.
EarnedWorkHoursAtCutoffCumulative double never Work hours earned up to the cut-off.
EarnedWorkHoursAtCutoffCumulativeBaseline double never Obsolete. Earned hours against the baseline up to the cut-off. Progress is only reported against Live, so this carries no meaning; use EarnedWorkHoursAtCutoffCumulative.
ActualWorkHoursAtCutoffPeriodic double never Work hours expended within the cut-off period.
ActualWorkHoursAtCutoffCumulative double never Work hours expended up to the cut-off.
PlannedWorkHoursAtCutoffPeriodicBaseline double never Work hours planned within the cut-off period, baseline revision.
PlannedWorkHoursAtCutoffCumulativeBaseline double never Work hours planned up to the cut-off, baseline revision.
PlannedWorkHoursAtCutoffPeriodicBaselineOriginal double never Work hours planned within the cut-off period, original baseline revision.
PlannedWorkHoursAtCutoffCumulativeBaselineOriginal double never Work hours planned up to the cut-off, original baseline revision.
PlannedWorkHoursAtCutoffPeriodicCurrent double never Work hours planned within the cut-off period, current revision.
PlannedWorkHoursAtCutoffCumulativeCurrent double never Work hours planned up to the cut-off, current revision.
PlannedWorkHoursAtCutoffPeriodicRevised double never Work hours planned within the cut-off period, revised revision.
PlannedWorkHoursAtCutoffCumulativeRevised double never Work hours planned up to the cut-off, revised revision.
PlannedProgress double never Progress the activity was planned to have reached, as a percentage — 20 means 20%.
CurrentProgress double never Progress reported on the activity, as a percentage — 20 means 20%.
TotalFloatHours double never Hours the activity can slip before it delays the end of the schedule, Live.
PlannedProgressBaseline double never Planned progress in the baseline revision, as a percentage. An estimate, so this one is valid.
CurrentProgressBaseline double never Obsolete. Current progress against the baseline, as a percentage. Progress is only reported against Live, so this carries no meaning; use CurrentProgress.
TotalFloatHoursBaseline double never Total float in the baseline revision. An estimate, so this one is valid.
ActivityType string never What kind of activity this is, Live — see the values below the table.
ReportScheduleId int32 never The report schedule this row belongs to.
OriginalBaselineActivityId int32 possible The original baseline planning activity this row draws on, into activity.parquet of that revision's snapshot.
BaselineActivityId int32 possible The baseline planning activity this row draws on, into activity.parquet of that revision's snapshot.
CurrentActivityId int32 possible The current-revision planning activity this row draws on.
RevisedActivityId int32 possible The revised-revision planning activity this row draws on.
ActivityId int32 possible The Live planning activity this row draws on, into activity.parquet.
ActivityTypeBaseline string possible Activity type in the baseline revision — same values as ActivityType.
Label string possible Human-readable name of the activity.
LocalSystemId string possible Identifier of the activity in the planning system it was imported from.
GlobalSystemId string possible Identifier of the activity in the system of record it originates from.
Code string possible Short code identifying the activity in the source planning system.
IlapId string possible Stable ILAP identifier for the activity, carried across imports and deployments.
BaseUri string possible Base URI recorded alongside the ILAP identifier when the activity was imported.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

ActivityType and ActivityTypeBaseline are one of: NotSet, RegularActivity, MilestoneStart, MilestoneFinish, Hammock, TaskDependent, ResourceDependent, LevelofEffort, WBSSummary.

Note the two hour families. AtCutoffPeriodic is the amount falling within the cut-off period; AtCutoffCumulative is the amount up to and including the cut-off. Adding one to the other double-counts.

This file carries metadata columns. See Metadata columns.

reportActivityPeriodized.parquet

One row per activity per point in time: the time-phased hours series. It holds no activity dates at all — what it shows of a date movement is the consequence, the planned-hours curve shifting along the time axis, never the dates themselves.

Column Type Empty? Meaning
ActivityId int32 never The report activity this series belongs to — into reportActivity.parquet, not activity.parquet, despite the name.
PointInTime timestamp never The point on the time axis this row reports.
ActualHours double possible Hours expended at this point in time.
EarnedHours double possible Hours earned at this point in time.
TotalPlannedHours double possible Total planned hours for the activity, Live.
PlannedHours double possible Hours planned at this point in time, Live.
PlannedLoad double possible Planned load at this point in time, Live.
CalendarHours double possible Working hours the calendar makes available at this point, Live.
RemainingHours double possible Hours still to be expended at this point, Live.
TotalMinusActualHours double possible Total planned hours less hours expended, Live. The only variant of this that carries meaning.
OriginalBaselineTotalPlannedHours double possible Total planned hours, original baseline revision.
OriginalBaselinePlannedHours double possible Hours planned at this point, original baseline revision.
OriginalBaselinePlannedLoad double possible Planned load at this point, original baseline revision.
OriginalBaselineCalendarHours double possible Calendar hours at this point, original baseline revision.
OriginalBaselineRemainingHours double possible Remaining hours at this point, original baseline revision.
OriginalBaselineTotalMinusActualHours double possible Obsolete. Original baseline planned hours less hours expended. The hours expended are Live, so this mixes a baseline estimate with Live actuals; use TotalMinusActualHours.
BaselineTotalPlannedHours double possible Total planned hours, baseline revision.
BaselinePlannedHours double possible Hours planned at this point, baseline revision.
BaselinePlannedLoad double possible Planned load at this point, baseline revision.
BaselineCalendarHours double possible Calendar hours at this point, baseline revision.
BaselineEarnedHours double possible Obsolete. Earned hours against the baseline at this point in time. Progress is only reported against Live, so this carries no meaning; use EarnedHours.
BaselineRemainingHours double possible Remaining hours at this point, baseline revision.
BaselineTotalMinusActualHours double possible Obsolete. Baseline planned hours less hours expended. The hours expended are Live, so this mixes a baseline estimate with Live actuals; use TotalMinusActualHours.
CurrentTotalPlannedHours double possible Total planned hours, current revision.
CurrentPlannedHours double possible Hours planned at this point, current revision.
CurrentPlannedLoad double possible Planned load at this point, current revision.
CurrentCalendarHours double possible Calendar hours at this point, current revision.
CurrentRemainingHours double possible Remaining hours at this point, current revision.
CurrentTotalMinusActualHours double possible Obsolete. Current-revision planned hours less hours expended. The hours expended are Live, so this mixes a current-revision estimate with Live actuals; use TotalMinusActualHours.
RevisedTotalPlannedHours double possible Total planned hours, revised revision.
RevisedPlannedHours double possible Hours planned at this point, revised revision.
RevisedPlannedLoad double possible Planned load at this point, revised revision.
RevisedCalendarHours double possible Calendar hours at this point, revised revision.
RevisedRemainingHours double possible Remaining hours at this point, revised revision.
RevisedTotalMinusActualHours double possible Obsolete. Revised-revision planned hours less hours expended. The hours expended are Live, so this mixes a revised-revision estimate with Live actuals; use TotalMinusActualHours.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

The unprefixed columns are Live; OriginalBaseline, Baseline, Current and Revised prefix their revision. ActualHours has no revision variants at all, because it is an actual. EarnedHours and TotalMinusActualHours are the two that do carry variants they should not — BaselineEarnedHours because earned hours are only reported against Live, and the four TotalMinusActualHours variants because they subtract Live actuals from a non-Live estimate. All five are marked obsolete above.

This file carries no metadata columns.

reportResource.parquet

Resources in reporting form, with the planning resource of each revision carried as its own column. One row per resource per report schedule.

Column Type Empty? Meaning
ReportScheduleId int32 never The report schedule this row belongs to.
ResourceId int32 possible The Live planning resource this row draws on, into resource.parquet.
BaseLineResourceId int32 possible The baseline planning resource this row draws on. Note the capital L.
OriginalBaselineResourceId int32 possible The original baseline planning resource this row draws on.
CurrentResourceId int32 possible The current-revision planning resource this row draws on.
RevisedResourceId int32 possible The revised-revision planning resource this row draws on.
Code string possible Short code identifying the resource in the source planning system.
IlapId string possible Stable ILAP identifier for the resource, carried across imports and deployments.
BaseUri string possible Base URI recorded alongside the ILAP identifier when the resource was imported.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

This file carries no metadata columns. Its planning counterpart resource.parquet does — if you are looking for a resource metadata field, it is there.

reportCalendar.parquet

Calendars in reporting form, with the planning calendar of each revision carried as its own column. One row per calendar per report schedule.

Column Type Empty? Meaning
ReportScheduleId int32 never The report schedule this row belongs to.
CalendarId int32 possible The Live planning calendar this row draws on, into calendar.parquet.
BaseLineCalendarId int32 possible The baseline planning calendar this row draws on. Note the capital L.
OriginalBaselineCalendarId int32 possible The original baseline planning calendar this row draws on.
CurrentCalendarId int32 possible The current-revision planning calendar this row draws on.
RevisedCalendarId int32 possible The revised-revision planning calendar this row draws on.
Code string possible Short code identifying the calendar in the source planning system.
IlapId string possible Stable ILAP identifier for the calendar, carried across imports and deployments.
BaseUri string possible Base URI recorded alongside the ILAP identifier when the calendar was imported.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

This file carries no metadata columns.

reportProfile.parquet

Distribution profiles in reporting form, with the planning profile of each revision carried as its own column. One row per profile per report schedule; the curve itself stays in profilePoint.parquet.

Column Type Empty? Meaning
IlapId string possible Stable ILAP identifier for the profile, carried across imports and deployments.
ReportScheduleId int32 never The report schedule this row belongs to.
ProfileId int32 possible The Live planning profile this row draws on, into profile.parquet.
BaseLineProfileId int32 possible The baseline planning profile this row draws on. Note the capital L.
OriginalBaselineProfileId int32 possible The original baseline planning profile this row draws on.
CurrentProfileId int32 possible The current-revision planning profile this row draws on.
RevisedProfileId int32 possible The revised-revision planning profile this row draws on.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

This file carries no metadata columns.

reportStructure.parquet

Breakdown structures in reporting form, with the planning structure of each revision carried as its own column. One row per structure per report schedule.

Column Type Empty? Meaning
ReportScheduleId int32 never The report schedule this row belongs to.
StructureId int32 possible The Live planning structure this row draws on, into structure.parquet.
BaselineStructureId int32 possible The baseline planning structure this row draws on.
CurrentStructureId int32 possible The current-revision planning structure this row draws on.
RevisedStructureId int32 possible The revised-revision planning structure this row draws on.
OriginalBaselineStructureId int32 possible The original baseline planning structure this row draws on.
Label string possible Human-readable name of the structure.
LocalSystemId string possible Identifier of the structure in the planning system it was imported from.
GlobalSystemId string possible Identifier of the structure in the system of record it originates from.
Code string possible Short code identifying the structure in the source planning system.
IlapId string possible Stable ILAP identifier for the structure, carried across imports and deployments.
BaseUri string possible Base URI recorded alongside the ILAP identifier when the structure was imported.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

This file carries no metadata columns.

reportStructureElement.parquet

Structure nodes in reporting form, with the planning node of each revision carried as its own column. One row per node.

Column Type Empty? Meaning
ReportStructureId int32 never The reporting structure this node belongs to, into reportStructure.parquet.
StructureElementId int32 possible The Live planning node this row draws on, into structureElement.parquet.
BaselineStructureElementId int32 possible The baseline planning node this row draws on.
CurrentStructureElementId int32 possible The current-revision planning node this row draws on.
RevisedStructureElementId int32 possible The revised-revision planning node this row draws on.
OriginalBaselineStructureElementId int32 possible The original baseline planning node this row draws on.
Label string possible Human-readable name of the node.
LocalSystemId string possible Identifier of the node in the planning system it was imported from.
GlobalSystemId string possible Identifier of the node in the system of record it originates from.
Code string possible Short code identifying the node in the source planning system.
IlapId string possible Stable ILAP identifier for the node, carried across imports and deployments.
BaseUri string possible Base URI recorded alongside the ILAP identifier when the node was imported.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

Unlike structureElement.parquet, this file carries no parent column — the hierarchy is held on the planning side.

This file carries no metadata columns.

reportActivityStructureElement.parquet

Which reporting activities hang off which structure nodes. One row per membership.

Column Type Empty? Meaning
ReportActivityId int32 never The reporting activity, into reportActivity.parquet.
StructureElementId int32 never The structure node it hangs off.

Like its planning counterpart, this file carries no Id and no Created.

This file carries no metadata columns.

reportResourceAssignment.parquet

Resource assignments in reporting form, with the planning assignment of each revision carried as its own column. One row per assignment.

Column Type Empty? Meaning
IlapId string possible Stable ILAP identifier for the assignment, carried across imports and deployments.
ReportActivityId int32 never The reporting activity the resource is assigned to, into reportActivity.parquet.
ReportResourceId int32 never The reporting resource assigned, into reportResource.parquet.
ReportProfileId int32 possible The reporting profile spreading the quantity over time, into reportProfile.parquet.
ResourceAssignmentId int32 possible The Live planning assignment this row draws on, into resourceAssignment.parquet.
BaseLineResourceAssignmentId int32 possible The baseline planning assignment this row draws on. Note the capital L.
OriginalBaselineResourceAssignmentId int32 possible The original baseline planning assignment this row draws on.
CurrentResourceAssignmentId int32 possible The current-revision planning assignment this row draws on.
RevisedResourceAssignmentId int32 possible The revised-revision planning assignment this row draws on.
Code string possible Short code identifying the assignment in the source planning system.
BaseUri string possible Base URI recorded alongside the ILAP identifier when the assignment was imported.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

This file carries no hours or progress at all — those live on reportActivity.parquet and reportActivityPeriodized.parquet. It also carries no metadata columns, though its planning counterpart resourceAssignment.parquet does.

reportSuccessor.parquet

Activity links in reporting form, with the planning link of each revision carried as its own column. One row per link.

Column Type Empty? Meaning
IlapId string possible Stable ILAP identifier for the link, carried across imports and deployments.
ReportActivityId int32 never The predecessor reporting activity the link runs from, into reportActivity.parquet.
SuccessorReportActivityId int32 possible The successor reporting activity the link runs to.
SuccessorId int32 possible The Live planning link this row draws on, into successor.parquet.
BaseLineSuccessorId int32 possible The baseline planning link this row draws on. Note the capital L.
OriginalBaselineSuccessorId int32 possible The original baseline planning link this row draws on.
CurrentSuccessorId int32 possible The current-revision planning link this row draws on.
RevisedSuccessorId int32 possible The revised-revision planning link this row draws on.
Type string never Which ends of the two activities the link joins — see the values below the table.
ReportCalendarId int32 possible The reporting calendar the lag is measured against, into reportCalendar.parquet.
Id int32 never Identifier of the row, unique within this snapshot only.
Created timestamp never When the row was created in ILAP Analytics.

Type is one of: StartToStart, StartToFinish, FinishToStart, FinishToFinish.

The lag itself is not carried here — it stays on successor.parquet.

This file carries no metadata columns.

See also

Was this article helpful?

That’s Great!

Thank you for your feedback

Sorry! We couldn't be helpful

Thank you for your feedback

Let us know how can we improve this article!

Select at least one of the reasons
CAPTCHA verification is required.

Feedback sent

We appreciate your effort and will try to fix the article