Tagged: Scale Toggle Comment Threads | Keyboard Shortcuts

  • Gerrit Eicker 07:00 on 7. November 2012 Permalink
    Tags: , , , , , , , , Scale,   

    Metaverse and Scalability 

    Metaverse scalability: permitting concurrent, efficient use by massive numbers of identities; http://eicker.at/MetaverseResearch

  • Gerrit Eicker 15:12 on 13. January 2012 Permalink
    Tags: , Agility, , Amazon Elastic MapReduce, , , Apache Hadoop, , , , Big Data Processing, , Business Opportunities, , , , , , , Data Science, , , , , , , , , Microsoft Hadoop, , , , , Scale, , ,   

    Big Data 

    Larger datasets allow better predictive analytics: Big Data is a lot more than business buzz; http://eicker.at/BigData

    • Gerrit Eicker 15:12 on 13. January 2012 Permalink | Reply

      Wikipedia: “In information technology, big data consists of datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing. This trend continues because of the benefits of working with larger and larger datasets allowing analysts to ‘spot business trends, prevent diseases, combat crime.’ Though a moving target, current limits are on the order of terabytes, exabytes andzettabytes of data. Scientists regularly encounter this problem in meteorology, genomics, connectomics, complex physics simulations, biological and environmental research, Internet search, finance and business informatics. Data sets also grow in size because they are increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing), software logs, cameras, microphones, Radio-frequency identification readers, wireless sensor networks and so on. Every day, 2.5 quintillion bytes of data are created and 90% of the data in the world today was created within the past two years.

      ATD: “Over the last several years, there has been a massive surge of interest in Big Data Analytics and the groundbreaking opportunities it provides for enterprise information management and decision making. Big Data Analytics is no longer a specialized solution for cutting-edge technology companies – it is evolving into a viable, cost-effective way to store and analyze large volumes of data across many industries. … Big Data technologies like Apache Hadoop provide a framework for large-scale, distributed data storage and processing across clusters of hundreds or even thousands of networked computers. The overall goal is to provide a scalable solution for vast quantities of data … while maintaining reasonable processing times. … The barriers to entry for Big Data analytics are rapidly shrinking. Big Data cloud services like Amazon Elastic MapReduce and Microsoft’s Hadoop distribution for Windows Azure allow companies to spin up Big Data projects without upfront infrastructure costs and allow them to respond quickly to scale-out requirements. … To apply this new technology effectively, it is important to understand its role and when and how to integrate Big Data with the other components of the data warehouse environment. … Hadoop provides an adaptable and robust solution for storing large data volumes and aggregating and applying business rules for on-the-fly analysis that crosses boundaries of traditional ETL and ad-hoc analysis. It is also common for the results of Big Data processing jobs to be automated and loaded into the data warehouse for further transformation, integration and analysis. … Big Data adoption will continue to be driven by large and/or rapidly growing data being captured by automated and digitized business processes. … As we move into the age of Big Data, companies that are able to put this technology to work for them are likely to find significant revenue generating and cost savings opportunities that will differentiate them from their competitors and drive success well into the next decade.”

      ORR: “Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it. .. The hot IT buzzword of 2012, big data has become viable as cost-effective approaches have emerged to tame the volume, velocity and variability of massive data. … The value of big data to an organization falls into two categories: analytical use, and enabling new products. … The emergence of big data into the enterprise brings with it a necessary counterpart: agility. Successfully exploiting the value in big data requires experimentation and exploration. Whether creating new products or looking for ways to gain competitive advantage, the job calls for curiosity and an entrepreneurial outlook. … If you could run that forecast taking into account 300 factors rather than 6, could you predict demand better? – This volume presents the most immediate challenge to conventional IT structures. … The importance of data’s velocity – the increasing rate at which data flows into an organization – has followed a similar pattern to that of volume. … A common theme in big data systems is that the source data is diverse, and doesn’t fall into neat relational structures. It could be text from social networks, image data, a raw feed directly from a sensor source. None of these things come ready for integration into an application. … The phenomenon of big data is closely tied to the emergence of data science, a discipline that combines math, programming and scientific instinct. Benefiting from big data means investing in teams with this skillset, and surrounding them with an organizational willingness to understand and use data for advantage. … If you pick a real business problem, such as how you can change your advertising strategy to increase spend per customer, it will guide your implementation. While big data work benefits from an enterprising spirit, it also benefits strongly from a concrete goal.

      Beye: “What is This Thing Called Big Data? – It’s difficult to avoid big data these days. More correctly, it’s difficult to avoid the phrase ‘big data.’ It has become such an integral part of the sales pitches of so many vendors and the blog posts of so many experts that one might be forced to conclude that big data is all-pervasive. The truth is far more complex. Even a definition of big data is elusive. … Big data is not, in essence, something entirely new. The problem is, to a large extent, one of scale; hence the name. However, the insights we currently have into these categories listed earlier and the different tools and approaches they require must be carried forward into how we handle these same data categories at a larger scale. … As a result, depending on your point of view, big data appears either as a giant wave of business opportunity or a huge precipice of potential technological and management pain.

      Beye: “What is the Importance and Value of Big Data? – Recognizing that big data has long been with us allows us to look at the historical value of big data, as well as current examples. This allows a wider sample of use cases, beyond the Internet giants who are currently leading the field in using big data. This leads us to the identification of value in two broad categories: pattern discovery and process invention. … Clearly, discovering a pattern in, for example, customer behavior may be very interesting, but the real value occurs when we put that discovery to use by changing something that reduces costs or increases sales. … More recently, combining data sets from multiple sources, both related and unrelated, with increasing emphasis on computer logs such as clickstreams and publicly available data sets has become popular. … The second approach to getting business value from big data involves using the data operationally to invent an entirely new process or substantially re-engineer an existing one. … For those contemplating investment in big data, the most important conclusion from this article is to recognize that there are very specific combinations of circumstances in which big data can drive real business value. Sometimes, of course, it is the price for simply staying in the game…

Compose new post
Next post/Next comment
Previous post/Previous comment
Show/Hide comments
Go to top
Go to login
Show/Hide help
shift + esc