Continuing the great cleanup. It looks like archiveProcessor will have to go (and merge with Archive)

Moving Transitions logic from plugin class to API

Cleaning up some separation of concerns

NOTE: I'm breaking the Multisites API here, removing the _evolution. we should re-fix it...

such as removing duplicate code and a lot of refactoring, the code is now much more readable!

Killing the Archiver from VisitFrequency. Nice! using VisitsSummary.get API instead, with a visitorType==returning segment
Fixing a 4 year old bug which I'm happy about!
Also removing an old hack in Archive.php

Removing what seems to be unused code? @sgiehl you added it but I cant find any code calling generateDatatable.

Introducing new PluginsArchiver class. Plugins can create an Archiver inheriting this abstract class.

Factoring out archiving Custom variables + cleaning it up (breaking down huge archiveDay function into several methods)

Renaming some core code to make sense (sumMetrics and makeEmptyRow instead of ugly names)

TODO: continue with other plugins

Refactored archive data querying code. Removed tree-like data structure from Piwik_Archive, added memory optimizations for archiving (not storing archive processing instances), added optimization for querying data indexed by site + date, added ability to select multiple blobs/metrics at once (from different plugins), added optimization to Piwik_ArchiveProcessing_Period where numerics/records are selected all at once instead of one at a time.

Also added data access layer object for archive querying code, allowed anonymous function objects to be used as DataTable filters, and removed 'timestamp' DataTable metadata.

Also fixing the "look ahead code" to lookahead at most the visit standard length

Adding failing test case showing the bug (visit over several days, creates too many visits) refs #3957

Make sure provider segment suggestion values use non-prettified provider values, add tests for getProvider report & provider segment, and fix build.

Proper fix for this problem, adding static array of "Row evolution not supported"

Also I decided to enable tests so they all run on my box /standard ubuntu, rather than travis CI.
Fixed a bug in tests that plugins language files were not loaded early enough.

Enabling PDF tests on php 5.4 only, rather than 5.3, so I can create the test files on my computer and have green travis - hopefully that works!

 * making tests pass!
 * enabling DevicesDetection tests, even though DevicesDetection plugin is disabled by default.

 * making some tests pass == fixing bugs

Adding new DevicesDetection plugin, developed by Clearcode and sponsored by a client of Piwik Professional Services. A beautiful work of engineering, all released under GNU/GPL license!

Fixes #3612
 * when enabled, the plugin will add a new submenu: Visitors> Devices
 * the new devices report contains NEW reports:
  * Much improved Device type (tracking 'car browser', 'console', 'desktop', 'feature phone', 'smartphone', 'tablet', 'tv')
  * Device brand (tracking more than 100 brands such as Nokia, Nintendo, Lenovo or Apple.
  * Device model (tracking hundreds of phone/console models)
  * Operating System versions (tracking 70 variations of operating systems including Ubuntu vs Kubuntu vs Debian vs Lubuntu vs Xubuntu)
  * Operating System families (Android vs Google TV vs Windows vs Windows mobile vs Mobile gaming consoles)
  * Browser versions
  * Browser families

Refs #3505 There is some basic TV detection included and maybe you can help contribute better detections (see the .yml data files containing the regular expressions in YAML format)

HOW DOES IT WORK

This is quite beautiful system. It is a plugin that disabled by default. when enabled, it will create additional columns in the DB. Also at tracking, it will look at the user agent, and try to match it against one browser we know in the databases. The databases of user agent matching are composed by 3 YML files, parsed by spyc.php into php array.

These 3 YML took dozens of hours of work and testing with dozens of mobile phones and devices for accuracy. We are happy with the result as they should cover > 80% of the devices commonly used. We hope the community will help us build up these YML files and make them better, so we can track accurately 90% or 95% of requests.

The performance overhead is pretty small, but parsing the YML files + running dozens of regex will add some overhead. This is why it is still disabled by default. We will think about how to integrate it in core, in the next few months.

Please let me know if you find any problem with this new awesome code!

Hiding row evolution graph URL from output for Referrer.getAll
Please double test it's all working!

Fixes #3940 Adding '&format=php' to ensure the data within the bulk request, is converted only once and is valid JSON. Also works in XML (not in CSV).

I updated the bulk request doc at: http://piwik.org/docs/analytics-api/reference/#toc-advanced-users-send-multiple-api-requests-at-once