Wednesday, August 28, 2013

Thread Safe ObservableCollection in WPF

The following is an improvement on the implementation found by Jonathan. Firstly it runs each event handler on the dispatcher associated with it rather than assuming that they are all on the same (UI) dispatcher. Secondly it uses BeginInvoke to allow processing to continue while we wait for the dispatcher to become available. This makes the solution much faster in situations where the background thread is doing lots of updates with processing between each one. Perhaps more importantly it overcomes problems caused by blocking while waiting for the Invoke (deadlocks can occur for example when using WCF with ConcurrencyMode.Single).

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.Linq;
using System.Text;
using System.Windows.Threading;

namespace ThreadSafeApplication.Helper
{
  public class ThreadSafeObservableCollection<T> : ObservableCollection<T>
  {
    public override event NotifyCollectionChangedEventHandler CollectionChanged;
    protected override void OnCollectionChanged(NotifyCollectionChangedEventArgs e)
    {
      NotifyCollectionChangedEventHandler collectionChanged = this.CollectionChanged;
      if (collectionChanged != null)
        foreach (NotifyCollectionChangedEventHandler handler in collectionChanged.GetInvocationList())
        {
          DispatcherObject dispatcherObject = handler.Target as DispatcherObject;
          if (dispatcherObject != null)
          {
            Dispatcher dispatcher = dispatcherObject.Dispatcher;
            if (dispatcher != null && !dispatcher.CheckAccess())
            {
              dispatcher.BeginInvoke(
                  (Action)(() => handler.Invoke(this,
                      new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Reset))),
                  DispatcherPriority.DataBind);
              continue;
            }
          }
          handler.Invoke(this, e);
        }
    }
  }
}


Reference: http://stackoverflow.com/questions/2137769/where-do-i-get-a-thread-safe-collectionview

Thursday, August 1, 2013

Palo- An Open Source Business Intelligence Solution

Palo- An Open Source Business Intelligence Solution                                                                                                                                                                                                               


Akanksha Rathore , Shweta Joshi


Abstract-The past Business Intelligence market was strictly dominated by closed source and commercial tools but according to today's market scenario the business users are highly influenced by business intelligence decisions and often choose data discovery products in addition to/as the traditional business tools. The availability of open source business intelligence tools has expanded the scope of BI platforms in future. An Open Source BI platform provides a full spectrum of BI capabilities which reduces the overhead for the development and management of each application, and lets the user feel like he/she is using a single BI solution. This paper defines the open source business intelligence platform along with introduction to PALO an open source business information platform.
Keywords: open source, business intelligence tools, OLAP server, Excel
I.        INTRODUCTION
The increasing focus on open-source software has reached the mainstream business intelligence market. As some organizations are looking to reduce costs in their sometimes large Business Intelligence deployments, they are hoping that open source gives them greater leverage for their money.  Initially the commercial platforms were commonly considered superior to Open Source ones but, Now the Open Source Business Intelligence platforms have evolved much faster than commercial ones since they are not constrained by compatibility problems and rigid architectures.
This section of the paper provides an introduction to open source software and open source business intelligence platform. The section two defines the PALO platform which is a complete open source business intelligence platform.

A.       OPEN SOURCE SOFTWARE

Open source software is based around the idea that the users are allowed to work freely on the production or development of any software. Some experts define "open source" as a philosophical concept, others consider it a pragmatic methodology. Being very commonly used in software development, many experts consider that the development of open source software is a concept specific to this activity. In reality, open source can be applied in any area where a product is manufactured or developed. There are also examples of applications and technology, medicine, education, culture and more. Open Source software components are software that besides the execution of the necessary files, provides free source code and especially the freedom to let users modify the software as they wish.
Fig 1: Life cycle of development of Open Source Software

The open source model is a very practical way of developing software in a rapidly changing environment with many advantages like giving the power to control the software's code, lowers total cost of ownership, reducing dependence on software vendors, easier ways to customize etc. The figure 1 above shows the life cycle of open source software.

B.       OPEN SOURCE BUSINESS INTELLIGENCE PLATFORM

Business intelligence platforms are used by all types of users from IT staff to consultant to business users- to build applications that help organizations to learn about and understand their business. The term intelligence has been used by researchers in artificial intelligence since the 1950s. Business intelligence became a popular term in the business and IT communities only in the 1990s.Gartner defined business intelligence platform as a software platform that delivers 14 capabilities [4] and these are organized in 3 categories. These are as follows:
1)      Integration: this category has four capabilities-
·         Business intelligence infrastructure
·         metadata management
<![if !supportLists]>·         <![endif]>Development tools


<![if !supportLists]>·         <![endif]>Collaboration
<![if !supportLists]>2)       <![endif]>Information Delivery: this category has six capabilities-
<![if !supportLists]>·         <![endif]>Reporting
<![if !supportLists]>·         <![endif]>Dashboard
<![if !supportLists]>·         <![endif]>Ad hoc Query
<![if !supportLists]>·         <![endif]>Microsoft Office Integration
<![if !supportLists]>·         <![endif]>Searched Base BI
<![if !supportLists]>·         <![endif]>Mobile BI
<![if !supportLists]>3)       <![endif]>Analysis: This category has four capabilities-
<![if !supportLists]>·         <![endif]>Online Analytical Processing(OLAP)
<![if !supportLists]>·         <![endif]>Interactive Visualization
<![if !supportLists]>·         <![endif]>Predictive Modeling and Data Mining
<![if !supportLists]>·         <![endif]>Scorecards
The emergence of open source business intelligence tools suites can configure itself as a cheaper alternative, with equivalent reliability and functionality levels, compared to available commercial solutions, especially in a context dominated by economical and financial crisis. A scientific paper focusing on OS BI, proposed by Thomsen and Pedersen [3] considers the functionalities available in single tools but it does not consider BI platforms. The Open Source Business Intelligence platform can adopt two different models-

<![if !supportLists]>·         <![endif]>Commercial open source: This model provides for an open source product that meets the user's basic needs (i.e. community edition); an enterprise edition of the product can be purchased and it usually includes enhanced features as well as support and training services.
<![if !supportLists]>·         <![endif]>Free and Open Source Software (FOSS): The product is completely free, no enterprise solution is available, and thus all the functionalities are available to the community for free.

Fig 2: Reference Architecture for Open Source BI platform

The figure 2 above shows Open Source BI platform provides a full spectrum of BI capabilities within a unified system that reduces the overhead for the development and management of each application, and lets the user feel like he/she was using a single BI solution.
<![if !supportLists]>II.      <![endif]>PALO
PALO is a complete open source business intelligence suite. The Palo BI Suite combines four core BI applications - Palo OLAP Server, Palo Web, Palo for Excel and ETL Server - into one comprehensive and customizable BI platform. This products represents a high-end Business Intelligence solution which is available entirely free of any license fees. Jedox AG, headquartered in Freiburg (Germany) with offices in Great Britain and France, is one of the leading suppliers of Open-source Business Intelligence and Corporate Performance Management solutions in Europe. Jedox' core product, Palo BI Suite, accommodates the entire range of BI requirements including planning, reporting and analysis. [5]
Fig 3: Palo Architecture
The figure 3 above shows the architecture of Palo suite. The user interface is designed with regard to technical know-ledge level of the business user. It is designed to make the product responsive. Response times for screen actions should be very fast to get the same user experience as in products like Microsoft Excel. Palo's user management is also much simpler than most database applications, while still providing powerful data security features. The following section defines the core applications of PALO suite.
<![if !supportLists]>A.       <![endif]>PALO OLAP SERVER
Palo OLAP (online analytical processing) Server, the core component of core Jedox BI offering is a multidimensional in-memory database Server and typically used as a business intelligence tool for controlling and budgeting purposes with spreadsheet software acting as the user interface. Beyond the multidimensional data concept, Palo enables multiple users to share one centralized data storage. The most rewarding aspect of the in-memory technology is the speed advantage. Speed is important for Palo, as it supports multidimensional formulas and write-back of data. When the data in the cube changes, all aggregated numbers change as well as all calculation results are instantly available real-time. The cubes do not have to be reprocessed. They are ready to use as soon as the users defines them. The Palo OLAP Server also supports the Palo Supervision Add-In for workflow capabilities, event handling and scripting.
Palo OLAP Server also supports real-time modeling. This means users can modify hierarchies and even create new cubes on the fly in the familiar Excel environment. The Palo OLAP Server has an advanced design that is far more powerful than a traditional OLAP technology. In addition to the traditional aggregation functionality of OLAP cubes Palo also offers a sophisticated calculation engine that supports cross-dimensional business rules.
Palo OLAP offers a variety of ways to extend application via API and standard serial interfaces. Serial interfaces in VBA, PHP, C++, Java and .NET allow Palo OLAP to be extended. In this way partners and customers are able to provide further functionalities for Palo OLAP.
<![if !supportLists]>B.       <![endif]>PALO WEB
Palo Web includes all tools and modules that are necessary to create comprehensive Planning, Analysis and Reporting applications. All components are 100% web-based and do not need to be installed on PC.
Palo Web consist of Palo spreadsheet that offers the comfortable User interface and usability known within Microsoft Excel, but also adds new features such as a dynamic formatting technology which simplify creation of dynamic Reports and Planning; A Palo Pivot allows Ad-Hoc requests on all Databases of a Palo OLAP Data model. It allows filtering of all Dimensions, as well as their rows and columns. A request can be saved at any point and be shared with any collaborators; A Palo User Manager manages all users and user groups within the Palo Suite that are allowed to access single components and Databases; The Palo ETL Manager can create supervise and launch all Data integration projects. The Palo ETL Manager also creates monitoring reports of the launched Data integration processes; The Palo File manager allows you to set user rights for every file within the Palo suite in addition to the Palo Pivot and Palo Spreadsheet files. It offers every user and every application a data space to save files on. Hence, other information that may be included in a text doc or a PDF can easily be integrated into the Palo Application.
Fig 4: Palo Frameset manager
 Palo Web 3.2 introduces a new document type called Frameset. With Framesets it is possible to arrange several Palo Spreadsheets into one single application; split screen in different areas and use those areas to load and display different workbooks. This functionality can be used, for example to extend application navigation. The figure 4 above shows the file manager menu to customize frameset.
<![if !supportLists]>C.      <![endif]>PALO FOR EXCEL
Microsoft Excel is the most widely used BI-Tool that provides a cell based system based on Excel formulas that direct­ly access the database. This allows users to create a wide variety of reports without the usual limitations of a single view on the data that nearly all other BI systems have. The Palo Excel Add-in can be used for all scenarios from simple reports to be shared among a few colleagues up to complete applications for large audiences of end users. It offers build-in functionality for importing data from files or relational databases into a Palo OLAP Server. One method is the internal loop function of the data import wizard, which allows non-technical users to apply the power of spreadsheet calculations to the problem of data import. This spreadsheet approach is ideal for business users who need ad-hoc imports because it is simple and leverages the user's know-ledge of spreadsheet functions.
Palo is cell based i.e. when we execute a query in a relational database, the smallest result we get is one record, usually consisting of several fields. When we run a query in Palo, our result is a single cell value. Palo thinks in cells, not in records, and this is the reason why Palo is so compatible with Microsoft Excel, because Excel is cell-based, too. Palo for Excel works with Microsoft Office XP, 2003, 2007 and 2010. We can also use Palo for Open Office Calc.
Fig 5: Palo web Spreadsheet
The Palo for excel 3.2 version have many additional features like set of new refreshed style sheets are provided to show data in paste view, a Palo for excel spreadsheet can be published as a Palo Web Spreadsheet, it is also possible to open any Palo Web Spreadsheet in Microsoft Excel, to view  or modify it. The figure 5 above shows the web spread sheet for Palo. A supervision server event wizard is also available that provides easy ways to pre-create login and cube area.
<![if !supportLists]>D.      <![endif]>PALO ETL SERVER
The ETL-Server provides industrial strength extraction of mass data from a wide range of data sources. It transforms and loads both master data and transactional data into a wide range of target databases. The ETL-Server not limited to loading the Palo OLAP Server, but it has been tailored to the specific requirements of mass data imports and exports to and from Palo models. Here ETL means Extraction, Transformation and Loading of Mass Data from Heterogeneous Sources. All common relational databases are suitable source systems and can be connected to via a standardized interface. Even complex transformations and aggregations can be modeled with Palo ETL Server.
Fig 6: Work Flow in ETL Server
Palo ETL Server is based on a platform-independent Client-Server architecture offering high scalability and multi-user ability. The figure 7 below shows the architecture of Palo ETL Server. Palo ETL-Server can be used in two different modes: In stand-alone mode as a simple desk­top application or in Client-Server mode where an ETL service is installed on the server machine which manages all ETL projects centrally and executes them on demand.
Fig 7: ETL Server Architecture
As source systems, all common relational databases such as Oracle, Microsoft SQL Server or MySQL are supported. The connection is established via a JDBC driver. Palo OLAP Server itself can be used as a source system; transformation processes between two Palo models and export to flat files are possible. The ETL-Server provides advanced modeling features for hierarchies. It has detailed support for defining consolidations, attributes and weights. Palo ETL-Server itself features a SOAP based ETL-API interface, which allows open access to all modeling, monitoring and execution functionalities. Palo ETL Server is easily extendable to meet customer specifications. Complex transformations that exceed the capabilities offered by standard transformations can be implemented directly in Java.

<![if !supportLists]>III.   <![endif]>CONCLUSION

This paper shows that the open source phenomenon can become a valid alternative to commercial platforms within the BI context .It also shows that the OS BI platforms provide an added value with respect to single BI tools since they allow several functionalities to be accessed transparently and a set of processes to be centralized and simplified thus reducing the administration and development effort.
Palo makes available a remarkable number of BI functionalities even if it adopts an open source model. Users are quickly able to create a web-based reporting, and to publish the reports. it provides a secure saving and publishing of reports and the installation of Palo is also easy.
REFERENCES

<![if !supportLists]>1]       <![endif]>OpenI (2011) „Open Intelligence, http://www.openi.org/

<![if !supportLists]>2]       <![endif]>Business Intelligence: One Version of the Truth ComputerWorld, Julia  King, 2003-12-22. 
<![if !supportLists]>3]       <![endif]> Thomsen, C., Pedersen, T. B.: A Survey of Open Source Tools for Business Intelligence . In DaWaK, pp. 74–84, 2005.
<![if !supportLists]>4]       <![endif]> Gartner research, http://www.gartner.com/technology

<![if !supportLists]>5]       <![endif]>Palo (2011) „Open Source Business Intelligence, http://www.jpalo.com/en/
<![if !supportLists]>6]       <![endif]> Osterloh, M., and Rota, S. G. 2007. "Open Source Software Development – Just Another Case of Collective Invention?," Research Policy (36:2), pp. 157-171.





Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment.