(Translated by https://www.hiragana.jp/)
Google Ads Developer Blog: dfp_reports

After our last round of spring cleaning, we've gone back to the drawing board to take another look at how we could make reporting better. There currently is an abundance of Dimensions, DimensionAttributes, and Columns (and more coming with each release), so in an effort to simplify the list of fields, we will be sunsetting the following reconciliation-related dimension attributes / columns in all versions. This will happen on September 1, 2015.

After our last round of spring cleaning, we've gone back to the drawing board to take another look at how we could make reporting better. There currently is an abundance of Dimensions, DimensionAttributes, and Columns (and more coming with each release), so in an effort to simplify the list of fields, we will be sunsetting the following reconciliation-related dimension attributes / columns in all versions. This will happen on September 1, 2015.

Columns with equivalent replacements:

Columns without equivalent replacements:

While the first two have equivalent replacements, the latter ones are not being replaced, as they don’t exist in core product reporting either.

This means that usage of these columns / dimension attributes will begin throwing errors in all versions starting September 1st. If your network is actively using any of these, please update your reports to switch to the supported fields or remove them entirely. If you have any questions, comments, or concerns about this, you know where to reach us!

Now that it’s spring again (in the Northern Hemisphere at least), it’s time for DFP’s annual spring cleaning! In this edition, we’ll be doing some pruning of our ReportService. What does this mean for you? We’re sunsetting some reporting dimensions, attributes, and metrics in existing versions (before the version is fully sunset), so your reports will break if you don’t migrate before the shutoff dates. I know what you’re wondering: “should I panic?”. Absolutely not. This type of behavior rarely occurs, so as long as you phase out usage for these particular fields, you should be fine moving forward.

Now that it’s spring again (in the Northern Hemisphere at least), it’s time for DFP’s annual spring cleaning! In this edition, we’ll be doing some pruning of our ReportService. What does this mean for you? We’re sunsetting some reporting dimensions, attributes, and metrics in existing versions (before the version is fully sunset), so your reports will break if you don’t migrate before the shutoff dates. I know what you’re wondering: “should I panic?”. Absolutely not. This type of behavior rarely occurs, so as long as you phase out usage for these particular fields, you should be fine moving forward.

Merged Metrics

Remember when Doubleclick for Publishers was called DART? I, too, get nostalgic about our old ad server, but it’s been a couple of years since we transitioned to the new DFP platform, and it’s just about time when the merged reporting columns are no longer useful (these columns only existed so you could continue reporting on delivery that spanned DART and DFP). In all versions after v201502, we will no longer provide merged reporting columns and dimension attributes in the API, that is, anything starting with 'MERGED_' or contains '_LIFETIME_MERGED_.' After August 1, 2015, these columns and dimension attributes will stop returning data entirely and will return INVALID_COLUMNS in all versions that still include them.

There are three scenarios in which you’re using these columns:

  1. Just for fun.
  2. Because you forgot you’re using them.
  3. Because you have lifetime line items that have carried over from DART (in which case you’ll have to recreate these). To give you an example, if the metric you care about is impressions, you can get the DART delivery portion by subtracting the portion of delivery from DFP Premium (AD_SERVER_IMPRESSIONS) from the MERGED value (MERGED_AD_SERVER_IMPRESSIONS) which represents the aggregate DART and DFP Premium volume. Additionally, you should make the switch to the non-merged columns and dimension attributes as soon as possible.

Dimension Filters

But wait, there’s more! Our next API version (v201505) will be the last to support some of our infrequently used dimensionFilters.

  • MOBILE_LINE_ITEMS
  • WEB_INVENTORY_UNITS
  • MOBILE_INVENTORY_UNITS
  • WHOLE_NETWORK
  • PARTNER_STATS_TYPE_ESTIMATED
  • ACTIVE_ADVERTISERS
  • PARTNER_STATS_TYPE_RECONCILED
  • WEB_LINE_ITEMS
  • ALL_SALESPEOPLE

In each of the cases above, the filters either no longer provide meaningful information (as is the case with mobile vs. web line items and ad units with platform unification complete), or weren’t being used at all.

Similar to the changes above, after August 1, 2015, these dimension filters will return an INVALID_DIMENSION_FILTERS error in any version that still includes them.

So if you’re using any of the reporting features above, consider this an early heads up (and an opportunity) to refactor some of your code for spring cleaning.

As usual, if you have any questions, comments, or concerns, don’t hesitate to let us know on the forums.

In the coming weeks, we will be deprecating all ACTIVE_VIEW_NOT* report columns in v201405, v201403, v201311 and v201306 of the DFP API. These columns are no longer supported in the DFP query tool, and the DFP API is following suit. The following columns will be affected:

Column.TOTAL_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.TOTAL_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.AD_SERVER_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.AD_SERVER_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.ADSENSE_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.ADSENSE_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.AD_EXCHANGE_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.AD_EXCHANGE_ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Column.ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS
Column.ACTIVE_VIEW_NOT_MEASURABLE_IMPRESSIONS

Migration
Normally all features are supported until the API version is sunset. This deprecation is a rare case where these report metrics are being disabled for existing versions due to product changes related to viewability. If you are using v201408 or later, you will not be affected by this deprecation. If you are currently using these columns in v201405 or earlier, you can replace them with their logical opposites. Alternatively, to retain the same metrics, you can calculate them from the logical opposite and the rate. For example, Column.TOTAL_ACTIVE_VIEW_NOT_VIEWABLE_IMPRESSIONS is equivalent to

Deprecation errors
If you do not migrate, your reports will return the following error:

ReportError.COLUMNS_NOT_SUPPORTED_FOR_REQUESTED_DIMENSIONS

If you have any questions or migration troubles, please reach out to us on our developer forum.

A lot of our DFP API developers have been asking recently about how to filter report data by custom targeting key ID. Currently the DFP API allows you to filter report data by custom targeting value ID only. Until we have official support for filtering by custom targeting key ID in reports, you can use the CustomTargetingService and the ReportService together to achieve this goal.

A lot of our DFP API developers have been asking recently about how to filter report data by custom targeting key ID. Currently the DFP API allows you to filter report data by custom targeting value ID only. Until we have official support for filtering by custom targeting key ID in reports, you can use the CustomTargetingService and the ReportService together to achieve this goal.

Step 1: Use CustomTargetingService to get your keys and values

You will first need to use getCustomTargetingValuesByStatement and filter by the custom targeting keys you’re interested in to obtain all the corresponding values. For example:

    WHERE customTargetingKeyId IN (17, 18, 19)

If you have a lot of keys and values in your network, a better approach is to store these in a local database and do nightly syncs. Use getCustomTargetingKeysByStatement to obtain all the keys in your network, and then iterate through them, calling getCustomTargetingValuesByStatement for each key to obtain their values. Our client libraries all have examples of this. For instance, the Java example can be found in our ads Java client library GitHub repository. This way, you can look up the values associated with a custom targeting key more quickly and not do an additional API call.

Step 2: Use the values in a report query

Once you have the values for the custom targeting key you’re interested in, you can then use the ReportQuery.statement with the PQL IN clause to filter on the CUSTOM_TARGETING_VALUE_ID dimension. For example, say you were interested in filtering on custom targeting key ID of 7777. You look up the values of 7777 in Step 1, which gives you three value IDs: 3211, 88990, 123456. You can then use the IDs to effectively filter your report data by custom targeting key ID 7777.

    WHERE CUSTOM_TARGETING_VALUE_ID IN (3211, 88990, 123456)

However, please be aware that if you have a lot of custom targeting value IDs to filter on, you should batch them by querying for no more than 500 IDs at a time in the PQL IN clause. For example, you will run your report filtering on the first 500 IDs you’ve collected and save that report. Then you will run the same report on the next page of 500 IDs you’ve collected and so on until you have no more IDs. You can then combine the reports locally so that you have all the data for those custom targeting IDs.

If you have any questions about this, feel free to drop us a line on the DFP API forums or Ads Developer Google+ page.

In the recent DFP API releases, we announced the addition of more tables to the PublisherQueryLanguageService, starting with Line_Item and Ad_Unit. These tables are an alternative to retrieving entities from their respective services’ get***ByStatement methods. They allow you to retrieve sparse entities containing only the fields you’re interested in. For example, the following select statement retrieves the first page of only the ID and name of line items that are missing creatives.
SELECT Id, Name from Line_Item WHERE IsMissingCreatives = true LIMIT 500 OFFSET 0
In this blog post, we’ll go over some situations where this feature can be utilized to speed up entity retrieval times from hours to minutes.

Entity synchronization


The first major use case that benefits from these new tables is entity synchronization. For example, if you’re synchronizing line items on your network into a local database, you’re most likely using LineItemService.getLineItemsByStatement and hopefully taking advantage of the LineItem.lastModifiedDateTime field to only filter out line items that have changed since the last time you synchronized. But even with lastModifiedDateTime, this synchronization can still take a while, depending on how many line items you have on your network, and how complex their targetings are. If you don’t need to synchronize all the fields in your line item objects, you may be able to use the Line_Item PQL table to perform this synchronization instead.

If you do need to synchronize fields not yet available in the Line_Item table, such as targeting, you can still take advantage of this table for computed fields that don’t affect lastModifiedDateTime, such as LineItem.status. What you can do is synchronize your line items as usual with getLineItemsByStatement filtering on lastModifiedDateTime. Then update your local statuses with selected line item statuses from the Line_Item table (a very quick process):
SELECT Id, Status from Line_Item LIMIT 500 OFFSET 0

Match tables for reports


Local copies of line item information can also be used as match tables to construct more detailed reports. Sometimes, you may want more information in your reports than what is currently available as a dimensionAttribute. For example, if you run a report by line item ID, you may also want other line item information like isMissingCreatives to show in the report. Because LineItem.isMissingCreatives is unavailable as a DimensionAttribute, you can create a local match table containing line item IDs and additional columns to be included in the report. Then you can merge this match table with the report by the line item ID to obtain a report with those additional columns.

For example, let’s say you run a report with the following configuration:
Dimension.LINE_ITEM_ID
DimensionAttribute.LINE_ITEM_COST_TYPE
Column.AD_SERVER_IMPRESSIONS
The report in CSV_DUMP format looks something like this:
Dimension.LINE_ITEM_ID, DimensionAttribute.LINE_ITEM_COST_TYPE,
    Column.AD_SERVER_IMPRESSIONS
1234567, CPM, 206
1234568, CPD, 45
1234569, CPD, 4
To also include LineItem.isMissingCreatives in the report, you would fetch a match table and save it (as a CSV file for example) by retrieving ID and isMissingCreatives from the Line_Item table.
SELECT Id, IsMissingCreatives from Line_Item LIMIT 500 OFFSET 0
Full examples of how to fetch match tables are available in all our client libraries. For instance, Python’s is here. Then using a script or a spreadsheet program, merge the match table with the report to produce something like this:
Dimension.LINE_ITEM_ID, DimensionAttribute.LINE_ITEM_COST_TYPE,
    Column.AD_SERVER_IMPRESSIONS, LineItem.isMissingCreatives
1234567, CPM, 206, true
1234568, CPD, 45, false
1234569, CPD, 4, false
If you have any questions on these new PQL tables, or suggestions on what PQL tables you want in the next release, please let us know on the API forum, or on our Google+ Developers page.

Today we are launching v201308 of the DFP API, which brings many new and exciting features and a glimpse of the API’s future. This release improves report stability, offers a brand new way to fetch line items through a Publisher Query Language table, the ability to create first party audience segments, and the ability to see contending line item in forecasts. A detailed list of these features and what’s changed can be found on our release notes page.

Reporting

First off, we heard you loud and clear - reports are very important to you and when a report fails for no apparent reason, it’s incredibly frustrating. Starting today, we are taking major steps towards our goal to fix this. You’ll now notice that large reports, which would otherwise time out or fail with a 502 HTTP status code while fetching the download URL, will now spend more time preparing the report in the runReportJob stage. Some reports may still be too large to run, but any report that runs in the UI will now work via the API as well. We've also made this change behind the scenes, so you’ll start seeing improvements right away without having to switch to v201308. While we know there is still more work to be done, we hope this is a clear sign that we take this issue seriously and are working hard to improve it.

In addition to stability improvements, in v201308, we are launching two highly requested features: targeted criteria reporting and ActiveView (a.k.a. viewability metrics) columns. These features are not available in all networks yet, but you or your third-party will be able to use them as soon as they are rolled out, if your network is eligible.

Publisher Query Language

We are launching two major PQL features today - the LIKE keyword and the Line_Item table, both of which will be made available in all versions.

The LIKE keyword allows you to do wildcard matching for fields. For example, if you pass the filter statement “WHERE Name LIKE 'my order%'” to the OrderService.getOrdersByStatement method, it will match all orders that have a name beginning with ‘my order’ (like ‘my order 1’, ‘my order 2’ and ‘my orders’).

The other exciting feature of this release is the experimental Line_Item table. With this new table, you’ll be able to select only the fields you want for line items using the PublisherQueryLanguageService. For example:

SELECT Id, Name FROM Line_Item WHERE IsMissingCreatives = TRUE LIMIT 1000 OFFSET 0

This allows for extremely efficient synchronization; tasks that would take hours with LineItemService, will now take minutes. We think this will be a great fit for pulling “match tables” and we’ll have a follow-up blog post soon about how to do this. Although we are launching this with a limited set of fields, we have made it a priority to add more in upcoming releases and we’d love to hear your feedback on our forum or Ads Developer Google+ page. If you want to get started playing with these new features now, you can always visit the dfp-playground. Try using the Publisher Query Language section with a query like “SELECT Id, Name FROM Line_Item WHERE name LIKE 'Line Item #%' LIMIT 100”.

Last, but not least

Starting in v201308, we are adding support for creating first party audience segments with the AudienceSegmentService as well as retrieving contending line items with the ForecastService. We know the latter has been a long time coming, so we are looking forward to any feedback.

As always, if you have any suggestions or questions about the new version, feel free to drop us a line on our Ads Developer Google+ page.


 - , DFP API Team

Recently we announced how we’re helping publishers grow with DFP small business. This included making a number of features available to small business networks. In this blog post, we’ll describe how these changes affect the way small business networks use the API.
Recently we announced how we’re helping publishers grow with DFP small business. This included making a number of features available to small business networks. In this blog post, we’ll describe how these changes affect the way small business networks use the API.

Creatives

Creatives can now be reused for small business networks. When using LineItemCreativeAssociationService.createLineItemCreativeAssociation, you can now use an existing creative for the LineItemCreativeAssociation.creativeId instead of having to copy the creative.

CustomCreatives and TemplateCreatives are now available to small business networks as well. The following creatives that were previously only for small business networks now come back as TemplateCreatives starting in v201306:
  • FlashExpandableCreative → TemplateCreative.creativeTemplateId of 10001160.
  • FlashPushdownCreative → TemplateCreative.creativeTemplateId of 10001400.
  • FloatingFlashCreative → TemplateCreative.creativeTemplateId of 10000560.
  • TextAdCreative → TemplateCreative.creativeTemplateId of 10000440.

Targeting and Delivery

All custom targeting matchTypes are now available to small business networks and can be set on a CustomTargetingValue object when creating custom targeting using the CustomTargetingService.

Since competitive exclusions and ad exclusions are now available to small business networks, you can use the LabelService to create and manage labels for them. They can then be set on the LineItem.appliedLabels and AdUnits.appliedLabels fields.

Reports

Reach reports are now available to small business networks. You can run reach reports via the API using the ReportService’s reach report columns and reach report date range types.

City-level geographic data has also been made available in reports for small business networks and can be added to reports using the city dimensions.

Network administration

Since custom user roles are now available to small business networks, if you create a custom user role using the web UI, you can use UserService.getAllRoles to obtain the IDs of those roles and assign them to your users.

If you’re a small business network and have any questions or concerns about how these changes may affect your API use, please post your questions to the DFP API forums. Additionally, the Ads Java client library’s DFP examples are a great place to start exploring these additional features.

We are pleased to announce the release of v201306 of the DFP API - one of our largest launches yet. It introduces many new features for creative management, forecasting, reporting, and includes a beta release of the sales management services. A detailed list of these features and what’s changed can be found on our release notes page.

Sales Management API

With the launch of v201306 also comes the release of the Sales Management beta services. Sales Management is currently being piloted by a small group of partners and we plan to extend availability throughout the year. We'd like to offer beta access to developers who plan to leverage the API to build connections to other business tools and systems.

The sales services allow you to:
For more information, take a look at our use cases guide.

To request access to the Sales Management services, please complete this form and our team will be in touch with further instructions.

Reporting

As we detailed in our previous blog posts, improving the stability and feature set of the reports is a top priority. Some highlights of note for v201306:
  • Columns, dimensions, and dimension attributes will now appear in reports in the order in which they were requested. This does not affect previous versions.
  • If you have the feature enabled, you can now specify a timeZone to run the report in. Supported time zones for your network can be found using the Time_Zone table through the PublisherQueryLanguage service. Please contact your account manager for questions regarding enabling this feature.

  • More than 50 columns, dimensions, and dimension attributes have been added to bring the API closer to parity with the UI. 
Creatives

To make managing non-standard creatives easier, beginning in v201306, the creative types FlashExpandableCreative, FlashPushdownCreative, FloatingFlashCreative, and TextAdCreative have been replaced with the type TemplateCreative. In addition, the TemplateCreative.destinationUrl field has been added for use with %u or %%DEST_URL%% macros. As more template types are added to DFP, our hope is that you’ll be able to take advantage of them with less code modification.

If you have any suggestions or questions about the new version, feel free to comment on our plus page: Ads Developer Google+ page.

 - , DFP API Team

In the recent DFP v201302 release, we announced that the ability to run sell through reports (STRs) is now available via the API. This has led to a bit of confusion as some developers who are able to run STRs through the web UI received a ...
In the recent DFP v201302 release, we announced that the ability to run sell through reports (STRs) is now available via the API. This has led to a bit of confusion as some developers who are able to run STRs through the web UI received a ReportError.COLUMN_VIEW_NOT_ALLOWED message when using these new dimensions through the API. We would like to clarify how you’ll be able to access STRs.

We are in the process of updating STRs for networks that have them enabled. The current version of STRs isn’t accessible via the API. Once updated, you will be able to run STRs through the API right away as the columns already exist in the report service. Please speak with your account manager about when your network will be updated.

At Google, we enjoy hearing from you, the developer community, and working with you to ensure that progress is being made. We think the latest DFP API release reflects positively on how we work better together and we're excited to announce version v201208. This release adds new types of creatives, support for optimization, rich media, and video interaction report columns, along with new options for downloading reports. A full list of improvements from this release can be found on our release notes page. We also want to remind you that we host virtual office hours via Google+ hangouts in order to make sure your voice is heard. Stay tuned to our Google Developers Live calendar to catch the next one on September 18th.

Reporting improvements

In v201208, we’ve added 64 new columns which enable you to pull metrics for optimization, rich media, and video. Using these new columns, you’ll now be able to better track performance of your network including determining the interaction time of your rich media (e.g. RICH_MEDIA_INTERACTION_TIME), locating videos which complete the most (e.g. VIDEO_INTERACTION_COMPLETE), or analyzing the revenue resulting from optimized impressions (e.g. OPTIMIZATION_OPTIMIZED_REVENUE). In addition to these columns, we’ve added the CREATIVE_TYPE dimension and the ability to include custom fields to help you better break down your reports.

For applications which cannot process gzip files, you can now download reports already deflated using the new ReportService.getReportDownloadUrlWithOptions method. If you choose to not use gzip compression, we still highly recommend that you set the HTTP header Accept-Encoding: gzip to speed up downloads. We’ve also added the ability to include report properties (e.g. network, user, generation date, etc...) and remove the totals row. If there are any other types of report download options you’d like to see, we’d love to hear about them on the forum.

Creative additions

For publishers who are using the cutting edge features of DFP, we’ve added support for four new creative types: AdSenseCreative, AdExchangeCreative, RichMediaStudioCreative, and AspectRatioImageCreative. AdSense and Ad Exchange creatives allow you to traffic line item level dynamic allocation ads by serving ad slot code snippets as the creative asset. Rich media studio creatives allow you to fetch creatives created using the DoubleClick rich media studio. Although these creatives are mostly used in a read-only manner (since they are created in the rich media studio and not DFP), some fields are mutable, such as the duration, any CSS overrides, and URLs. Finally, aspect ratio image creatives let you upload multiple image assets of the same aspect ratio to give you control of how images should be scaled; these creatives are mostly used in a mobile environment given the variety of screen sizes and resolution of today’s devices.

Last but not least

In our ongoing effort to bring the API up to parity with the UI, we’ve also added a number of smaller features. These include additional company partner types, the ability to set the mobile platform type on ad units, a friendly display string for inventory sizes, the option to associate line items with creative sets , and support for the recently released device category targeting criteria.

Our API and outreach efforts are constantly growing, but we can't do it without you. If there is anything you'd like to see us do better, please let us know or introduce yourself in the next Google+ hangout on September 18th.

 - , DFP API Team

Today we’ve released the newest version of the DFP API, v201206, which adds a significant number of reporting improvements. The new release also fully supports OAuth 2.0 as the authentication mechanism of choice and we encourage you to switch to OAuth 2.0 from ClientLogin or OAuth 1.0a. A full list of improvements from today’s release can be found on our release notes page.

Reporting improvements

In a few of our recent hangouts, we received the feedback that while our reports were great for generating important performance metrics, the CSV files that you downloaded were not always easily machine readable. To make it easier for you to consume reports, we’ve created a new ExportFormat - CSV_DUMP. Below is a list of the features of this new format:

  • Columns are now shown as Dimension.ENUM_VALUE or Column.ENUM_VALUE
  • All money values are now displayed in micro format in the currency of the network
  • All dates are now displayed as YYYY-MM-DD
  • All date-times are now displayed as YYYY-MM-DDThh:mm:ss±hh:mm
  • There is no "pretty printing" of values (i.e. commas) and there is no total row
You may also notice that the v201204 CSV export format has been replaced by CSV_EXCEL, which can be imported into Excel-like products.

As an important note to some of our developers, after upgrading to v201206, you will most likely need to update your code; many column names have changed to reflect a more accurate description of what metrics they are indeed pulling. For example, the column TOTAL_IMPRESSIONS has been changed to TOTAL_INVENTORY_LEVEL_IMPRESSIONS because the v201204 column could only be used with dimensions like AD_UNIT_NAME on the inventory level, i.e. it could not be used with line items, orders, companies or creatives. Alternatively, TOTAL_LINE_ITEM_LEVEL_IMPRESSIONS in v201206 should now be used with dimensions like LINE_ITEM_NAME and ORDER_NAME for instances where you need to include dynamic allocation impressions from AdSense or Ad Exchange line items. To determine how each column should be updated, visit the old column’s reference page and look for the phrase that begins with “Replaced with …”, e.g.

    Replaced with TOTAL_INVENTORY_LEVEL_IMPRESSIONS beginning in v201206.

Lastly, we’ve improved formatting for inventory reports that don’t use top level ad unit views. Most importantly, the duplicate columns clicks and impressions issue for hierarchical views has been fixed and the flat view report will now match how the report is downloaded from the UI.

OAuth 2.0

If you are an eagle-eyed developer, you may have noticed that we recently added OAuth 2.0 information to our authentication page. OAuth 2.0 is now fully supported in the DFP API and we are progressively adding support in our client libraries; Java, Python, .Net , and Ruby currently have full support, while PHP will very soon. In fact, our DFP test playground already uses OAuth 2.0 with the Java library. Please stay tuned to the project sites or the forum for announcements regarding future support.

Our next hangout is July 18th and we’ll be taking your report questions or anything else you might have on your mind. As always, let us know if you have any questions on our forum.

 - , DFP API Team