Agile Big Data and Many-Particle approach change Marketing and Sales effectiveness

Big data projects have broad impact on organizations. Big Data implementation overtakes
what normally Many-Particle data aggregation goodcould be considered a new way to conduct data management to business alignment. With Big Data the path from data sources to data intelligence changes drastically. The way to design and implement data intelligence definitively changed access, ingest, distil, processes and data visualization as well. Big data projects meet agile implementation, shorten the data intelligence lifecycle by increasing services capability and adequacy to fast-growing datasets, fast moving business. Accordingly, agile practice and many-particle approach minimize data entropy together with data access time cycles everywhere, preserve data security and enhance user experience to business instant realignment.

Contents
Introduction
Data Topology and Agile Big Data
The Many-Particle approach
Conclusion
Acknowledgment
References

Introduction
The way to move from today business data into Big Data intelligence could be a costly and time consuming process that could decrease the tremendous advantage of the Big Data and Cloud paradigms. Today, information is still misaligned with the business although the huge efforts of the past business intelligence projects: companies still use partial quantities of the real corporate data heritage. As a consequence, the data spectrum exploited is unpredictable and the process to align data and business is a long-term process. Agile Big Data aligns instantly data heritage and business data. Continuous data ingestion and distillation drastically reduces ETL process to run intelligence on the “big data-lake” when needed. Then, on-premise big data topology and functional data intelligence have a crucial role to meet profitability, customer affinity and fast moving business goals. This paper introduces the business case for Big Data to avoid Marketing and Sales data entropy, reduce risks and increase the likelihood of an aware and successful Big Data implementation.

Data Topology and Agile Big Data
Documenting data evolution and updating in the past could be considered a good practice in managing data. In the beginning of cloud paradigm, due to the cost cut down attraction, the practice to have a map of the company data heritage became a great benefit especially when services have to be subscribed in the cloud. Data models, a way to document the data heritage, evolved into MaaS (Model as a Service) that supports agile design and deliver of data services in the Cloud and makes the difference in planning a Big Data implementation project.

Considering data models doesn’t mean structured data only. On-premise models map data coming from structured, semi-structured and non-structured sources. Data models maps defined services topology would be moved on-premise or in the cloud. Still data models is needed for early exploration analysis and “ab-initio” services classifying parameters which define services boundaries (to personal cloud, financial parameters or healthcare positions, for example); data models (on SQL, No-SQL, Vectors or Graph structures) essentially doesn’t address the meaning the data have but identify the services’ classes before creating the data-lake. Of course, into the data-lake converge unusable data, unstructured or denormalized raw datasources as well. The more aware is the on-premise topology, the more secure and localizable is the big data usage both on-premise and in the Cloud. Further, agile MaaS approach reveals business process affected, operating requirements and stakeholders.

Big Data CorporateFig. 1 – Corporate Data-Lake and Agile Big Data approach

Accordingly, agile Big Data practice sets the link among on-premise data topologies and on-premise or in the cloud data intelligence. Topology leverages the company services asset into specific business objectives and will determine the successful user experience requirements and the proper rapid alignment with respect to the competitors.

This means that two crucial aspects have to be taken care:

  • Data is the “compass” to understand services capacity, stakeholders, culture of the organization: big data agility is based on data-driven approach. Therefore, in the incoming project setup minimize functional data behaviour. Use MaaS topology to define projects use cases data-driven. Data-driven project design defines data ingestion architecture and data landing into the data-lake and assist in understanding the best policy for continuous data feeding. Do not disregard this aspect: accurate data feeding is the core of Big Data approaches;
  • Move data analysis and functional aggregation to data intelligence applied on the data-lake. During ingestion and data landing data treatments have to be minimized Agile Big Data approach considers 2 zones: the in-memory one, based on data topology and on-premise supported by MaaS and data intelligence based on functional analysis and programming working on spare data.

Still, minimize any approach based on “ab-inizio” technology and software development. The Big Data ecosystem provides excellent platforms and MaaS agile approach helps to shift later the final technology choice/selection. Further, MaaS agile practice assists to clarify successes and failures zone and set expectations by time. This happens why when services have been set by on-premise topology then a link has been stretched among the data heritage and the data intelligence. There are no constraints between the raw data (documented or not) and the user experience that will leverage functional and business alignment. In the middle, only the data-lake exists, continuously changing and growing, continuously supplying information for the data intelligence ending.

The Many-Particle approach
Today, more of 70 percent of the world’s information is unstructured, not classified and, above all, misused: we are assisting to the greatest Marketing and Sales data myopia since they exist. Still, there is no awareness of the Big Data benefits for service and/or product companies, and again how the product’s companies can change their services based on goods production: great amount of data, exceptionally growing, high entropy, unknown correlations and limited data usage. The concept of on-premise topology introduces services as data-driven aggregation states applied to given parts of the data-lake. But this is what happens to many-particle system instability (yottabyte is 1024 byte with a binary usage of 280). Big data storages dimension near data-lake to many-particle systems. This vision destroys any traditional approach to Marketing and Sales.

If we consider the big data-lake, it contains fast moving content in order of data affinity and mass correlation. Depending upon dynamic data aggregation, data topologies may change by tuning on-premise data mapping. Consider data-lakes are mainly fed through:

– ingestion, distillation and landing from content based (datasources, datasets, operational and transactional DB’s);
– ingestion and distillation from collaborative feeding (dynamic collections of large amount of information on users’ behaviours coming from the internet, direct and/or indirect).

Collaborative ingestion can be managed as a content based as well in case of time needed to data intelligence ending has no strict constraints so to define a third method, the hybrid one.

This brief introduction tries to explain that the data-lake maps ab-initio topologies to services but also may classify more ecosystems the services are defined and applied to. Services live in the ecosystems and ecosystems depend upon data aggregation (why used, where used, how used, who uses) and just like aggregation states, big data density change dynamically. These changes are a consequence of datasources ingested, users experiences, customers behaviours, ecosystems interaction and, of course, business realignment. Marketing and Sales should change accordingly. But since data-lake may grow by 40 percent per year (in line with the estimation of the worldwide rate of information growth taking into account that unstructured data is growing 15 times faster than structured data – source IBM®), there is no way to get any (predictive) control for marketing and sales organization although data warehousing and/or sophisticated traditional data mining and analysis are in place.

Anyway, the data growth will be greater than ever in the next years and so the variance for data aggregation in the data-lake will have an exponential rising: this means many opportunities could be lost and again further marketing and sales entropy. Ab-initio topology by agile big data approach and functional programming applied to the data-lake supply the best answer for prescriptive analysis on many-particle big data systems. In fact, the data-lake allows to work on data cross-aggregation optimization, customer experience and aggregation states for services realignment with respect to the business ecosystems. Still, data-lake is an extraordinary real-time “what-if set” for prescriptive scenarios, data processing assumption and data risk propensity.

Data-Sea

Fig.2 – The Data-Lake is quickly becoming a Data-Sea with multi-particle-like data behaviour and dimension

Banking and Goods Production are 2 typical examples of Big Data agile implementation. Both are supplying services. Both are trying to align instantly and proactively offer and business changes. Banking and Financial services play a strategic role in relationship management, profitability performance to corporate groups, client companies and commercial banking networks. This is why financial applications need to be rapidly synchronized to ecosystems fluctuations states as ecosystem participants’ change everywhere their behaviour due to local and international business conditions. Functional big data paradigm working on many-particle data aggregation is prescriptive with respect to unpredictable services transition: it agilely realigns ecosystem services directions over on-premise data topologies mapping.

Goods production may tune services as a consequence of user’s experience by, for example, executing more focused and less time-consuming recommender systems. Goods production companies are in the run to provide personalized technical and commercial services, greater client loyalty and prescriptive offers starting soon when the clients interact or navigate the company website. With agile big data and many-particle approach, goods production potentially increases user similarity by data-lake massive data aggregations. Fast moving data aggregations constantly feed functional data intelligence to services realignment and topological correlations repositioning on-premise data similarities.

Two different paces, the same objective: be prescriptive, understanding “at earlier” which data aggregation state is the most proper along the data-lake instability and then contiguously realign products offer, services configuration and, consequently, keep ecosystems oversee: on-premise topology gauged on data-lake volume, data velocity and variety allows Marketing and Sales to tune on effective data aggregation to promptly adjust services to the ecosystem.

Conclusion
Client sentiment and user experience behaviour analytics allow rapid changes to product offerings or customer support which in turn enhance customer fidelity and business improvement. However data are growing exponentially and business alignment have to be provided in more decentralized environments. Agile MaaS approach based on data-driven raw volume, data velocity and variety together with on-premise services topology is a relatively low cost and light model. Topology does not influence data treatment. Data remains intact although services integrity and classification drive business, user experience and ecosystems alignment. Accordingly, agile practice and many particle approach we introduced minimize data entropy together with data access time cycles everywhere, preserve data security and enhance user experience to functional visualization realignment.

Acknowledgment
I sincerely thank Paolo La Torre for his precious feedback on contents and encouragement on publishing this paper. Paolo is working as Commercial, Technical and Compliance Project Supervisor for Big Data planning and engagement directions in finance and banking.

References
N. Piscopo, M. Cesino – Gain a strategic control point to your competitive advantage – https://www.youtube.com/watch?v=wSPKQJjIUwI
N. Piscopo – ID Consent: applying the IDaaS Maturity Framework to design and deploy interactive BYOID (Bring-Your-Own-ID) with Use Case
N. Piscopo – A high-level IDaaS metric: if and when moving ID in the Cloud
N. Piscopo – IDaaS – Verifying the ID ecosystem operational posture
N. Piscopo – MaaS (Model as a Service) is the emerging solution to design, map, integrate and publish Open Data
N. Piscopo – Best Practices for Moving to the Cloud using Data Models in the DaaS Life Cycle
N. Piscopo – Applying MaaS to DaaS (Database as a Service ) Contracts. An introduction to the Practice
N. Piscopo – MaaS applied to Healthcare – Use Case Practice
N. Piscopo – ERwin® in the Cloud: How Data Modeling Supports Database as a Service (DaaS) Implementations
N. Piscopo – CA ERwin® Data Modeler’s Role in the Relational Cloud
N. Piscopo – Using CA ERwin® Data Modeler and Microsoft SQL Azure to Move Data to the Cloud within the DaaS Life Cycle

Disclaimer – This document is provided AS-IS for your informational purposes only. In no event the contains of “Agile Big Data and Many-Particle approach change Marketing and Sales effectiveness ” will be liable to any party for direct, indirect, special, incidental, economical (including lost business profits, business interruption, loss or damage of data, and the like) or consequential damages, without limitations, arising out of the use or inability to use this documentation, regardless of the form of action, whether in contract, tort (including negligence), breach of warranty, or otherwise, even if an advise of the possibility of such damages there exists. Specifically, it is disclaimed any warranties, including, but not limited to, the express or implied warranties of merchantability, fitness for a particular purpose and non-infringement, regarding this document use or performance. All trademarks, trade names, service marks, figures and logos referenced herein belong to their respective companies/offices.

ID Consent: applying the IDaaS Maturity Framework to design and deploy interactive BYOID (Bring-Your-Own-ID) with Use Case

Introductionidentity

Current approaches to IDaaS on one hand enforce trust of consumer data using legal compliance, risk and impact assessment and the other hand require technical implementation of access controls to personal data held by an enterprise. Balancing trust has to be done across all layers, verifying person’s identities, showing the individual and the service is real, creating short term relationships and verifying and maintaining all long the Cloud service the user mapping between the enterprise and the cloud user account in a mesh federation. This makes sense only if enterprises design “on-premise” with MaaS their own flexible ID data model and can verify ID maturity and consistency before moving, and along, the ID service in the Cloud. Based on MaaS, the BYOID concept is a possible solution to ID models for consent policy design, management and deployment. The BYOID model is a means to expressing, tracing and updating consumer’s personal data policy requirements; however enterprise users’ privacy preferences are provided as well. The IDaaS Maturity Framework (IMF) defines and directs the BYOID practice. MaaS guide properties and personal preferences from the consent metamodel design to the ID deployment. Both ensure that ecosystem compliance is achieved and ID in the Cloud meets trustworthy relationships.

IMF supports flexible BYOID design and deployment

IDaaS is authentication and authorization infrastructure that is built, hosted and managed through different models by third-party service providers, resident in ID ecosystem frameworks. IDaaS for the enterprise is typically purchased as a subscription-based managed service.  One or more cloud service providers, depending upon the IDaaS model the enterprise deploys, may host applications and provide subscribers with role-based web access to specific applications or even entire virtualized infrastructure. IDaaS makes enterprises responsible in evaluating privacy risks and grade of confidence when moving the ID to the cloud. Accordingly, before externalizing the corporate IdM, consider the different IDaaS models are supported depending upon the maturity levels of:

– IdM/IAM system, in terms of implementation, maintenance and IdM/IAM governance capacity. ID, by its nature is de-centralized and then the maturity rank should consider the whole IdM/IAM system including data protection, data manageability, data security and organization ID awareness at all levels;

IMF BYOID Fluid LifecycleFig. 1 – An example of enterprise BYOID consent model lifecycle to IDaaS deployment and reconciliation

– SOA system, to really understand policies by applied processes’ de-coupling (privileges by user role, accreditations, de-accreditations …) and procedures dynamically acting into the organization;

– ID ecosystem reliability and adherence to the frameworks’ security criteria that measure service provider(s) compliance.

However, the levels of maturity gauged along the organization enables the enterprise to design its own ID as a consequence of the appropriate IDaaS model. The enterprise is able to bring in the ID ecosystem a configurable IDaaS model based on MaaS design to satisfy enterprise business rules. Business rules have impact on enterprise identity requirements and they balance and reconcile consumer identities needs. This “fluid” multiple-way enterprises-consumers solution, or BYOID, creates a high assurance level of ID ecosystem participants’ identities that could be used for enterprise access by respecting privacy and security requirements: IDaaS models contain BYOID properties and define “on-premise” BYOID maturity and consistency.

A new concept of ID consent: the BYOID fluid model

When registering to an Identity Platform, users would like represent themselves according to their behaviour having the option to approve selective or discretionary sharing of their private information and looking for the ability to obfuscate, mask or mesh some parts of personal data. So, ID platform and user are creating interactively a bond of trust as a part of the whole ID service. This is possible only if the consent of the individuals, the data protection conditions for processing their personal data and consent policies might be modelled “on-premise” by the enterprise IdM.

Looking at the IMF, the ID metamodel might sprout in the IdM/IAM maturity appraisal stage, according to the properties and requirements the enterprise needs to protect personal data and sensible information. The question now is the following: if the ID metamodel is designed in the company IdM, could the consent model be considered proprietary? The metamodel gathers the properties corresponding to the real enterprise requirements but it will be tested and appraised firstly in IdM/IAM system and then in the SOA maturity system. At that point features like interoperability, expression of functionality and user’s behaviour will be explicit aspects of the BYOID data model such as the following:

1)    Trust properties;
2)    Verification;
3)    Scalability and performance;
4)    Security;
5)    Privacy;
6)    Credential Types;
7)    Usability;
8)    Attributes;
9)    User Centricity/User Control.

The above properties are matter for the ID ecosystem public consent data model structure (basic/incoming tables of the BYOID metamodel). In the beginning, those metadata are properties of the company: the company’s BYOID metamodel. Once the BYOID metamodel has been defined, tested and approved as BYOID company data model, it will be released to the ID ecosystem as an IDaaS model subscription. Despite of different approach, each enterprise may then adopt and release his BYOID. Before deploying BYOID services in the Cloud, the BYOID model should be compared with other BYOID models already running into the ID ecosystem frameworks. To be accepted, BYOIDs have to meet a set of common requirements enforced by the consent public ID ecosystem framework authority: the more adaptive is the public consent model (continuously and rigorously improved), the more flexible, secure and reliable are the BYOIDs shared. It makes interactive, fluid and safe BYOIDs deployed through IDaaS. Still, this enables user’s behaviour can be captured both at high level (enterprise-ecosystem reconciliation) and at low level (personal-enterprise-ecosystem reconciliation). Therefore BYOID can be reconciled, renormalized and constantly trusted at all levels. Since BYOID metamodel contains the enterprise identity requirements, it might include and integrate the ID ecosystem identity properties and, if approved by the user (obligation to maintain the personal data securely), his personal properties. This aspect is very important: in fact, there’s significant risk for a company when both customer/user relationships and company data are stored on personal devices. Using BYOID deployed as an IDaaS subscription, company information is centralized based upon “on-premise” consent metamodels: this means that company information stored on personal devices is minimized and always centrally controlled.

BYOID Model Recon

Fig. 2 – Fluid BYOID update and reconciliation: IDaaS User Experience vs. BYOID IDaaS subscription

User’s personal properties might reside on the same company (central) metamodel/consent model or not depending upon user approval and, always possible, withdrawal (i.e. personal data should comply with data protection legislation and, where necessary, the approval of the individual must be obtained). In the figure 2 here is an example. In 1 the User tries a new behaviour (statistically relevant or as a recommender system outcome); in 2 the IDaaS user experience has to be changed and updated. Above we show 3 data models but in the MaaS representation they consist of a unique model containing the BYOID IDaaS subscription (master) that includes 2 sub-models: the company consent model and the user personal model. In 3, the consent model is modified to keep compliance with the company business rules/conduct mapped to the BYOID IDaaS subscription. In 4, finally the update is executed and the User might find his conduct as a new function. However, take note in the figure 2 a relational model-like formalism is applied. This is just a simplification. In effect, we are in a multi-level relational data model that can be represented with NoSQL, Vector or Graph DB else, depending upon the data analytics domain.

USE CASE: the fluid BYOID approach

Scenario

IDaaS models to move ID to the Cloud enable organizations to externalize identities data more knowingly and securely. Employees and customers behaviour changed: they continuously have business contacts, calls and meetings with personal devices. Since an increasing quantity of employees uses their mobile devices everywhere, identities can be resident and so associated to applications running on different framework in a multi-topology cloud configuration. What should be then the best IDaaS model satisfying this new employees/customers conduct? Could be managed all users, across multiple locations, while securing company data? Because of each identity may be managed by different identity management services, authentication and validation of identities by the cloud infrastructure could not be sufficient. Companies have to verify and control “on-premise” their ID maturity. BYOID based upon IDaaS models allows to identifying and securing identity properties. Further IDaaS models assist ID integrity control over shared topologies with a variety of ID ecosystem frameworks. IMF plays a crucial role in identifying the most appropriate IDaaS model before deploying the BYOID to the Cloud. Then the BYOID is an IDaaS model and can be designed “on-premise” and controlled along deployment and subscription.

Properties and Directions

This use case is concerned with enterprises deploying their BYOID in the Cloud using IDaaS models and IMF. There is a need for evaluating “on-premise” organization IdM/IAM and SOA maturity before moving the ID to the Cloud. Evaluating the organization maturity levels involves three steps:

  1. IdM/IAM maturity: measure the IdM/IAM maturity level;
  2. SOA maturity: measure SOA maturity level – policies (privileges by user role, accreditations, de-accreditations …) and processes dynamically acting;
  3. Identity Ecosystem reliability/maturity: measure the ecosystem maturity/reliability, and above all, the secure service continuity because in hybrid topologies identities may be owned by different cloud providers resident in multi-topologies.

Objectives are the following:

  • Enable organization to identify and set the best BYOID through IDaaS model based upon internals levels of IdM/IAM and SOA maturity compared to the ID ecosystem framework’s baseline adherence. This sets maturity in classifying the ID ecosystem framework and in evaluating the reliability the ID ecosystem may provide;
  • Deploy the proper BYOID model applying the correct subscription and adherence with respect to the IDaaS ecosystem;
  • Periodically measure the organization’s IdM/IAM and SOA maturity levels and verify the ID ecosystem reliability/maturity so to update, and eventually scale, the BYOID deployed.

However, accordingly with the objectives, the value of the ID ecosystem level of reliability/maturity is the outcome the company is expecting to:
–          Keep BYOID secure and controlled and supervise the IDaaS service subscription;
–          Contribute to the ecosystem as participant and/or as authority;
–          Be a participant/counterpart in setting and approving attributes providers, policies and relying party’s decisions and IDaaS ecosystem adherence;
–          Contribute to the IDaaS Trustmark definition and to the periodical appraisal and updating.

Tabella1Tabella21Tabella21

Table 1 – BYOID Use Case properties and directions

Process Flow along the IMF

Accordingly to this Use Case, the IMF process flow encompasses three steps:

Part 1: Appraise IdM/IAM Maturity Level – To cover definition, maintenance and upgrade of the organization IdM/IAM level of maturity. The IdM/IAM maturity value has to be periodically monitored and controlled to keep coherence with the IDaaS model deployed:

Use Case 1.1

Figure 3 – BYOID: IDM/IAM Maturity Level Appraisal

The Identity and Access Manager verifies the Maturity level of the IdM/IAM system:

  • The IdM Manager controls and regulates the accesses to information assets by providing policy controls of who can use a specific system based on an individual’s role and the current role’s permissions and restrictions. This ensures that access privileges are granted according to one interpretation of policy and all users and services are properly authenticated, authorized and audited;
  • The BYOID Manager reconciles BYOID metadata and update the BYOID metamodel.

The IAM Manager controls if users’ identities can be extended beyond corporate employees to include vendors, customers, machines, generic administrator accounts and electronic access badges, all ruled by policy controls.

Part 2: Appraise the SOA Maturity Level – To cover definition, maintenance and upgrade of the organization SOA maturity level. The SOA maturity level has to be periodically monitored and controlled to keep coherence with the BYOID released:

Use Case 1.2

Figure 4 – BYOID: SOA maturity level appraisal

The SOA Manager verifies the Maturity level of the SOA system through the SOA interoperability and defines the organization maturity in sharing services among departments:

  • The SOA Manager verifies that the map of communications between services is drawn starting from IdM/IAM system and achieved maturity
  • The SOA Manager controls and reports about the following crucial aspects:
  • SOA reference architecture achievements and evolution;
  • education to broaden SOA culture through the organization;
  • methods and guidelines that organization adopts to apply SOA;
  • policy for SOA appliance and governance.
  • The BYOID Practice Manager tests and executes BYOID consent model reconciliation based on metamodel reconciliation and update. If necessary, BYOID Manager renormalizes the consent model by roundtrip with the BYOID metadata at IdM/IAM maturity level.

Part 3: Appraise the ID Ecosystem Reliability/Maturity – To establish the maturity/ reliability of the ID Ecosystem Posture. The comparative maturity of BYOID (Company vs. ID Ecosystem participants vs. user preferences) has to be continually monitored: points of discontinuity, unmatched policies, and untrusted relationships have to be time by time acknowledged. This helps to better qualifying frameworks accountability, federation assets, and participants’ reliability and level of contribution:

Use Case 1.3

Figure 5 – BYOID: ID Ecosystem Maturity/Reliability Appraisal

The Service Manager verifies the Maturity/ Reliability level of the ID Ecosystem framework:

  • The Service Manager controls that contribution to the ecosystem by privacy aspects, security components and accountability mechanism settings are congruent
  • The Service Manager controls that common guidelines keep coherence with the company policies and standards strategy. Since more than a framework exists inside the ecosystem, rules to ensure that accreditation authorities validate participants’ adherence to the ecosystem requirements are to be verified and updated
  • The Service Manager controls adherence to the ID ecosystem of the IDaaS deployed to verify reliability and service continuity;
  • The Service Manager verifies that accreditation authority to ensure participants and frameworks are adherent to the identity ecosystem interoperability standards accepted
  • The Service Manager controls that the ID ecosystem contains all trusted frameworks that satisfy the baseline standards established and they are compliant with the company maturity level
  • The BYOID Practice Manager verifies the framework ecosystem common levels of adherence (baseline) and test and compare BYOID reliability properties;
  • The ID Ecosystem Management Service verifies BYOID adherence and security with respect the IDaaS subscription.

The ID Ecosystem Management service provides a combination of criteria to determine the service providers’ compliance among frameworks and ID ecosystem topologies: the combination defines policies, rules and, eventually, a Trustmark. It gives confidence to participants in deciding who to trust in terms of BYOID framework adherence and among all ID providers.

Conclusion

Managing digital identities across ID ecosystems frameworks is crucial to improve efficiency of business collaborations. Using everywhere personal devices is becoming a preferred conduct but before sharing the ID among cloud domains, all involved parties need to be trusted. Still, to meet the demanding needs of security, big data analytics and business intelligence, users and consumers need a more efficient and flexible paradigms. In this paper, we identify how BYOID fluid model satisfies on one hand company security and user data protection and, on the other hand, rapid updating and reconciliation to the user conduct. IMF provides the necessary platform for collaboration in ID ecosystem topologies. We introduce also a USE CASE to point out how BYOID built across ID company consent model and ID ecosystem trusted access model, can be a foundation to gauge and govern BYOID strategies. Further, the paper can be used to compare different BYOID IDaaS subscription to establish what maturity levels the company might support compared with all business partners running existing IDaaS maturity models and to ensure ID in the Cloud meets trustworthy relationships.

Acknowledgements

I have to sincerely thank Susan Morrow for the precious feedback on contents and Anil Saldhana for the useful comments on the IDaaS Maturity Framework.

References

N. Piscopo – IDaaS. Verifying the ID ecosystem operational posture
N. Piscopo – A high-level IDaaS metric: if and when moving ID in the Cloud
N. Piscopo – MaaS implements Small Data and enables Personal Clouds
N. Piscopo – Best Practices for Moving to the Cloud using Data Models in the DaaS Life Cycle
N. Piscopo – MaaS (Model as a Service) is the emerging solution to design, map, integrate and publish Open Data
N. Piscopo – MaaS applied to Healthcare – Use Case Practice
N. Piscopo – Applying MaaS to DaaS (Database as a Service) Contracts. An introduction to the Practice
N. Piscopo – Enabling MaaS Open Data Agile Design and Deployment with CA ERwin®
N. Piscopo – ERwin® in the Cloud: How Data Modeling Supports Database as a Service (DaaS) Implementations
N. Piscopo – CA ERwin® Data Modeler’s Role in the Relational Cloud
N. Piscopo – Using CA ERwin® Data Modeler and Microsoft SQL Azure to Move Data to the Cloud within the DaaS Life Cycle
N. Piscopo – Page 16 in Transform2, MaaS and UMA implementation

Disclaimer – This document is provided AS-IS for your informational purposes only. In no event the contains of “ID Consent: applying the IDaaS Maturity Framework to design and deploy interactive BYOID (Bring-Your-Own-ID) with Use Case” will be liable to any party for direct, indirect, special, incidental, economical (including lost business profits, business interruption, loss or damage of data, and the like) or consequential damages, without limitations, arising out of the use or inability to use this documentation, regardless of the form of action, whether in contract, tort (including negligence), breach of warranty, or otherwise, even if an advise of the possibility of such damages there exists. Specifically, it is disclaimed any warranties, including, but not limited to, the express or implied warranties of merchantability, fitness for a particular purpose and non-infringement, regarding this document use or performance. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies/offices.

MaaS implements Small Data and enables Personal Clouds

Abstract – MaaS (Model as a Service) sets a new concept to order and classify data modeling design and deployment to the Cloud. MaaS changes the way to move data to the Cloud because allows to define data taxonomy, size and contents. Starting from data model design, MaaS might guide the DaaS (Database as a Service) lifecycle, providing data granularity and duty rules: as a consequence, MaaS implements the new concept of Small Data.big-data_1

In fact, Small Data answers to the need of controlling “on-premise” data dimension and granularity. Anyway, Small Data is not data volume limitation. Small Data affords data modeling full configuration and provides 2 main advantages: data model scale and data ownership that provide assigned data deployment and, finally, data deletion in the Cloud.

Introduction

The inheritance coming from the past imposes to manage big data as a consequence of multiple integration and aggregation of data systems and data movement. Data coming from Social Networks intensive data applications have contributed to blow up the EB containers.

Administrating big data is not an option but rather a debt contracted above all with the data management history. Anyway, Big Data analytics seems to be a standard practice in storing massive data. Is there any way to change this norm? Companies have for many years used data models by designing and mapping data, started the change and today manage the “tiller” of their data heritage.

Accordingly, Small Data is far from a further catch-phrase: the antonyms with respect Big Data aims to pay attention in changing mind and using data models to design fit, noticeable data systems, especially when data are moved to the Cloud. MaaS implements Small Data, enables Personal Cloud and help to recover, to order and to classify, data inheritance of the past as well.

Why MaaS defines and implements Small Data
MaaS meets the ever-increasing need for data modeling and provides a solution to satisfy continuity of data design and application. Further, it helps in choosing and defining architectures, properties and services assets looking at the possible evolution and changes the data service can have. MaaS meets scaling and dimensioning in designing data systems, supports scalability and, in particular, agile data modeling appliance. Small Data starts when the dimension of the data system has to be defined and controlled since the description (metadata). In effect, we are improperly speaking of data systems: actually, we are dealing with data services. The Cloud is the great supplier of data services.

Since MaaS allows defining “on-premise” data design requirements, data topology, performance, placement and deployment, models themselves are the services mapped in the Cloud. In fact, data models allow to “on-premise” verify how and where data has to be designed to meet the Cloud service’s requisites. In practice, MaaS enables in designing the data storage model. The model should enable query processing directly against databases to strengthen privacy levels and secure changes from database providers;

– Modeling data to calculate “a priori” physical resources allocation. How many resources does the service need? Do database partitions influence resource allocation and/or replication? Modeling the data means designing the service; calculating a priori these magnitudes drives both deployment and database growth;

– Modeling data to predict usage “early” and to optimize database handling. Performance and availability are two of the main goals promised by the Cloud. Usage is not directly dependent upon the infrastructure and, as a consequence, could be a constraint. Calculating the usage rate means understanding the data application life cycle and then optimizing the data service properties;

– Designing multi-data structures to control databases elasticity and scalability. Models contain deployment properties and map the target database service. Therefore, the model designs “on-premise” database elasticity and scalability. Still, we will see later that multi-database design is a way to control data persistence.

Thus, imagine we have users asking for temporary services to be deployed and, after services have been closed, cancelled. Services satisfying these requirements are based upon data models designed by MaaS agile model techniques, controlled size and contents, fast update, rapid testing and continuous improvement forward the generation of the model/dataset to the target database. Models should set users’ data contents, dimension (for example, to suit to mobile services), data deployment (geo-location, timing …) and to be, on-demand, definitively destroyed.

With MaaS, Small Data can be defined since the metamodel design and then implemented, synchronized and deployed to the target by applying the DaaS lifecycle. Of course although, Small Data are placed under size and content control, they can be created and replicated infinitively: is this a further way to switch to Big Data again?

The following aspects would be considered:

1) Users should be enabled to destroy their Small Data in the Cloud when the service is over. This is a great feature for data security. Still, data navigation changes from infinite chains to point-to-point gaps (or better “Small to Small” Data) based upon Small Data models definition;

2) Time limit could be set or an over storage timing strategy might be defined for Small Data allocation in the Cloud;

3) Statistically speaking, by applying Small Data the average of data storing coming from intensive data applications (Social Networks, for example) should be computed and standard deviations estimated due to multiple storage allocation and, above all, free.

This doesn’t mean the YB is the last order of magnitude of data size the SI has defined. Definitively, MaaS enables service designers to plan, create and synchronize Small Data models from and to any web data source and data management system in the Cloud. Data granularity changes and enables designers to calibrate data models and data services by defining model’s content that allows users to deploy i.e. allocate and then, at the end of the service, to shred the data in the Cloud. This is a new frontier for Mobile services, Open Data and what today might be defined as Personal Cloud.

MaaS enables Personal Clouds
What is really new in the Small Data definition is the likelihood to move the data ownership from big players to users and, as a consequence, place under relation ownership and deployment location. Today data storage is almost under provider full control. Providers and storage players manage data users and data security. By applying MaaS, Personal Cloud is enabled and following here is what change:

1) Integrity defined into MaaS at the Small Data level is maintained through the service. Ownership matches the data structure/dataset deployed;

2) MaaS identifies trust boundaries throughout the IT architecture. Data models are the right way to define trust boundaries and ownership to prevent unauthorized access and sharing in “Small to Small” Cloud storage and navigation;

3) MaaS enables to set location in the data model. Any mismatch could be an infringement and must be reconciled with the terms registered in the Small Data design. Point to point navigation based upon Small Data simplifies Personal Data management and maintenance: this aspect has a positive impact on data storage order and security;

4) Ownership and data location are linked by relation. Once Personal Data in the Cloud has to be deleted, the Cloud Provider should assure the data are unrecoverable. Looking at data model mapping, data has to be destroyed in the location defined in the Small Data design. Data owners know where the data has been deployed because before opening the data service in the Cloud they may accept or not the location assigned and might ask for a new storage site.
Personal Cloud has large ranges of applications:

1) Mobile services in single/multiple personal storage by single/multiple dataset in the Cloud;

2) Healthcare services as introduced, for example, in [10] and in [11];

3) Open Data services, especially when they are interfaced to the above 1) and 2) services;

4) HR services, mainly when they concern curricula content. Owners should be free to definitively cancel personal data as defined in Small Data designing;

5) Generic personal data in the Cloud regardless they are permanent or temporary stored.

Applying MaaS, Small Data can be considered on-premise services because they collect behaviours and information concerning structures (to be deployed in the Cloud), access rights, security and scaling, partitioning and evolution. In other words, Small Data ensure that the behaviour and effectiveness of the released Cloud applications can be measured and tracked to meet user’s needs. Models leverage Cloud data services to enable flexible deployment and therefore enforce Personal Data persistence, storage and geo-location policies throughout the Cloud.

Conclusion
Big Data is a consequence, Small Data is a new start. MaaS provides Best Practices and guidelines to implement Small Data and Personal Cloud ownership by starting from data modeling and DaaS lifecycle. We want to underline that Big Data as we know is a consequence of how companies have for many years used, stored and maintained data. Small Data indeed might be a new way to manage data in the Cloud. Especially when personal data are considered, Personal Cloud provides, on one hand, a preconfigured and operational data definition (for example, local information vs. cloud information) and, on the other hand, the details of how to enable provisioning and deployment of multiple storage in the Cloud. Finally, starting from “on-premise” Small Data design, Personal Cloud can be applied and users can have soon an understanding of Cloud deployment, data centre geo-locations and service constraints.

Glossary
Big Data – Collection of datasets cannot be processed (analysis, storage, capture, search, sharing, visualization …) using on-hand database management tools or traditional data processing applications. It is due to data complexity, data amount and fast growth;
EB – Exabyte, unit of information or computer storage equal to one quintillion bytes (1018 bytes);
DaaS: Database as a Service;
MaaS: Model as a Service is a trade mark (MaaS);
SI – Système International d’unités metric prefix ;
YB – Yottabyte, unit of information or computer storage equal to one septillion bytes (1024 bytes).

References
[1] N. Piscopo – ERwin® in the Cloud: How Data Modeling Supports Database as a Service (DaaS) Implementations
[2] N. Piscopo – CA ERwin® Data Modeler’s Role in the Relational Cloud
[3] D. Burbank, S. Hoberman – Data Modeling Made Simple with CA ERwin® Data Modeler r8
[4] N. Piscopo – Best Practices for Moving to the Cloud using Data Models in the DaaS Life Cycle
[5] N. Piscopo – Using CA ERwin® Data Modeler and Microsoft SQL Azure to Move Data to the Cloud within the DaaS Life Cycle
[6] N. Piscopo – MaaS (Model as a Service) is the emerging solution to design, map, integrate and publish Open Data https://cloudbestpractices.wordpress.com/2012/10/21/maas/
[7] N. Piscopo – MaaS Workshop, Awareness, Courses Syllabus
[8] N. Piscopo – DaaS Workshop, Awareness, Courses Syllabus
[9] N. Piscopo – Applying MaaS to DaaS (Database as a Service) Contracts. An introduction to the Practice https://cloudbestpractices.wordpress.com/2012/11/04/applying-maas-to-daas/
[10] N. Piscopo – MaaS applied to Healthcare – Use Case Practice, https://cloudbestpractices.wordpress.com/2012/12/10/maas-applied-to-healthcare/
[11] N. Piscopo – MaaS and UMA implementation at page 16 in Transform2:, https://cloudbestpractices.files.wordpress.com/2013/01/transform-203.pdf
[12] Agile Modeling – http://www.agilemodeling.com/

Disclamer
“MaaS implements Small Data and enables Personal Clouds” (the Document) is provided AS-IS for your informational purposes only. In no event the contains of the document will be liable to any party for direct, indirect, special, incidental, economical (including lost business profits, business interruption, loss or damage of data, and the like) or consequential damages, without limitations, arising out of the use or inability to use this document or the products, regardless of the form of action, whether in contract, tort (including negligence), breach of warranty, or otherwise, even if an advise of the possibility of such damages there exists. Specifically, it is disclaimed any warranties, including, but not limited to, the express or implied warranties of merchantability, fitness for a particular purpose and non-infringement, regarding this document or the products’ use or performance. All trademarks, trade names, service marks and logos referenced herein belong to their respective companies/offices.

Cloud entrepreneurs: Seize a BOLD future

I accidentally stumbled on the cheesy acronym ‘BOLD’ when I was trying to think of the best way to capture the key ideas about the whole ‘data thing’, as I was thinking the essential points to highlight are that it’s about ‘Big Open Linked Data‘.

As we start to move more into definitions of ‘Cloud 2.0’, we start to look more at this aspect of an increasingly interconnected universe of data, in terms of what we think of as “the Cloud”, as much as we mean the virtualized hosting environments for running software, like Amazon.

The technologies and trends that underpin this environment include:

  • Big Data – As we start to generate data at the nano level of activity, like sensor readings, location tracking signals from vehicles or web click streams, the volume of it will become too big for normal databases to handle, requiring new Big Data technologies.
  • Open Data – Like the City of Toronto, some organizations will publish their data with licences that enable reuse and mash ups, enabling ecosystems of new apps.
  • Linked Data – Implementation of standards like XDI to enable “Linked Data“. This will enable universal updating of personal information, so that you’re not repeatedly updating your postal address each time you move for example.

In the same way the core standards like TCP/IP, DNS and HTML provided building block foundations for an interconnected web of content and simple services, these will do the same a next layer of more sophisticated, smart agent type apps.

For app developers this presents a Blue Ocean of opportunities, and also for hosting providers to operate the underlying infrastructure in the same way they currently run web and DNS servers for customers. For example Sungard recently made the smart move of announcing Hadoop hosting services.

Yes I know, a cheesy acronym but hey when I started in IT Sales I had to try and sell “BLOBS“. Yeah think about that.