Big Data
Larger datasets allow better predictive analytics: Big Data is a lot more than business buzz; http://eicker.at/BigData
Tagged: Processing Toggle Comment Threads | Keyboard Shortcuts
Gerrit Eicker
Google Search
Google: Another look under the hood of search – the evolution of Google Search; http://eicker.at/GoogleSearch
-
Gerrit Eicker
Google: “Over the past few years, we’ve released a series of blog posts to share the methodology and process behind our search ranking, evaluation and algorithmic changes. Just last month, Ben Gomes, Matt Cutts and I participated in a Churchill Club event where we discussed how search works and where we believe it’s headed in the future. – Beyond our talk and various blog posts, we wanted to give people an even deeper look inside search, so we put together a short video that gives you a sense of the work that goes into the changes and improvements we make to Google almost every day. While an improvement to the algorithm may start with a creative idea, it always goes through a process of rigorous scientific testing. Simply put: if the data from our experiments doesn’t show that we’re helping users, we won’t launch the change. … In the world of search, we’re always striving to deliver the answers you’re looking for. After all, we know you have a choice of a search engine every time you open a browser. As the Internet becomes bigger, richer and more interactive it means that we have to work that much harder to ensure we’re unearthing and displaying the best results for you.”
Google: “Following up on our video on how we make improvements to search, we wanted to share with you a short history of the evolution of search, highlighting some of the most important milestones from the past decade-and a taste of what’s coming next. – Our goal is to get you to the answer you’re looking for faster and faster, creating a nearly seamless connection between your questions and the information you seek. For those of you looking to deepen your understanding of how search has evolved, this video highlights some important trends like universal results, quick answers and the future of search. – For more information, go to Google.com/insidesearch”
SEL: “Google released a short video today highlighting some of its key milestones in search over the past decade. It’s both a fun blast from the past and a worthwhile reminder of how much things have changed over the years. The video is also a nice follow-on to the look under the hood of search that Google released in August.”
TC: “One anecdote centers on the attacks of September 11: in the wake of the attacks, many people were searching for ‘New York Twin Towers’ and related queries as they attempted to get the latest news – only to find that Google’s index didn’t have any relevant news stories because it was weeks old (Danny Sullivan has written more about this failure). Google’s quick-fix was to post links to relevant news articles on its homepage, and its stumble eventually led to the launch of Google News.”
ATD: “So, what would be a hard query that Google wants to answer in the future? Complex questions that take reasoning, says Google Fellow Amit Singhal. ‘In my ideal world, I would be able to walk up to a computer and say, ‘Hey, what is the best time for me to sow seeds in India, given that monsoon was early this year?’‘ Singhal says in the video.”
RWW: “4 Big Trends in the Evolution of Google Search … Univeral Search – Google’s introduction of universal search in 2007 was the beginning of a trend away from separating Web search results by type and toward putting it all in one place. … Google Goes Mobile and Local – Before long, Google was deep into the business of local commerce. With the rise of Android, Google had an end-to-end business of finding location-based results for local businesses, restaurants and destinations. … Google Search and Time – Google has changed the impact of time on search, as well as place. It has tweaked the way timeliness of content appears in search multiple times, and its latest update calculates when a search is probably looking for recent results rather than historical ones. … Google+: Google’s New Identity – Identity is the final piece of the puzzle. Google has personalized results for a while using Web history and sharing data. But with the launch of Google+, Google has introduced a form of social SEO. Social activity is now a fundamental part of how search results appear for users logged into Google’s ubiquitous Web services.“
Gerrit Eicker
Siri: Let’s Talk!
Potentially Apple’s Siri changes how we interact with computers entirely: Siri, let’s talk! http://eicker.at/Siri
bigL, 
Doll, 
Adrianna, and 6 others are discussing. -
Gerrit Eicker
Apple: “Siri. Your wish is its command. – Siri on iPhone 4S lets you use your voice to send messages, schedule meetings, place phone calls, and more. Ask Siri to do things just by talking the way you talk. Siri understands what you say, knows what you mean, and even talks back. Siri is so easy to use and does so much, you’ll keep finding more and more ways to use it. … Talk to Siri as you would to a person. Say something like ‘Tell my wife I’m running late.’ ‘Remind me to call the vet.’ ‘Any good burger joints around here?’ And Siri answers you. It does what you say and finds the information you need. And then it hits you. You’re actually having a conversation with your iPhone. … Siri not only understands what you say, it’s smart enough to know what you mean. So when you ask ‘Any good burger joints around here?’ Siri will reply ‘I found a number of burger restaurants near you.’ Then you can say ‘Hmm. How about tacos?’ Siri remembers that you just asked about restaurants, so it will look for Mexican restaurants in the neighborhood. And Siri is proactive, so it will question you until it finds what you’re looking for.”
Wikipedia: “Siri is a personal assistant application for iOS. The application uses natural language processing to answer questions, make recommendations, and perform actions by delegating requests to an expanding set of web services. The iOS app is the first public product by its makers, who are focused on artificial intelligence applications. Siri was acquired by Apple Inc. on April 28, 2010. – Siri’s marketing claims include that Siri adapts to the user’s individual preferences over time and personalizes results, as well as accomplishing tasks such as making dinner reservations and reserving a cab. … Siri was founded in December 2007 by Dag Kittlaus (CEO), Adam Cheyer (VP Engineering), and Tom Gruber (CTO/VP Design), together with Norman Winarsky from SRI’s venture group. … It was announced on October 4, 2011 that Siri will be included with the iPhone 4S. The new version of Siri is deeply integrated into iOS, and offers conversational interaction with many applications, including reminders, weather, stocks, messaging, email, calendar, contacts, notes, music, clocks, web browser, Wolfram Alpha, and maps. Currently, Siri only supports English (US, UK, and Australia), German and French. … Siri is a spin-out from SRI International’s Artificial Intelligence Center, and is an offshoot of the DARPA-funded CALO project, described as perhaps the largest artificial-intelligence project ever launched.”
TC: “The integration with iOS seems to be just as impressive as we’ve been hearing: you can ask it to remind you to call someone before you leave the office, and it’ll automatically create an entry in the Reminders app, complete with a geo-fence just to be sure. You can also ask Siri to read your queued messages to you and make an appointment in the Calendar app. – The worst part so far? Siri indeed seems to require the iPhone 4S’s extra horsepower, because it appears to be a 4S exclusive. The kicker? Siri was originally a run-of-the-mill iPhone app. What a shame. – Siri will be a beta for the time being, as it only supports English, German, and French voice input, but there are more language add-ons and tweaks to come.”
WP: “As rumored, Apple’s doing some all-new voice-control AI stuff in iOS 5. It’s called Siri, which is the name of the app Apple bought for $200 million a couple years ago. … You can also ask Siri to look things up on Wikipedia for you, and Siri can use Wolfram Alpha to do more complicated calculations. Siri’s list of capabilities is near endless, including asking it to play genres of music for you, look up something on maps, or what the weather is. Our favorite question? ‘Siri, who are you?’ Siri responds: ‘I am your humble personal assistant.’ … The bad news? All this great stuff is only available for the iPhone 4S – Apple had to do something to force an upgrade! In all seriousness, some of this AI functionality can be incredibly processor intensive, so Siri might be leaning on the A5 chip quite heavily.”
MLS: “Siri Search, makes use of Yelp’s business ratings, thus this makes instantly makes Yelp a strong local competitor to Google Places. Yelp is now very relevant to your small business rankings. Google Places has been the big dog in local optimization or as I call it, Local Awesomeization… And your places ranking and profile completion has become very important for your local marketing.- Now, Siri, which is a virtual assistant will be able to find you anything you want… and it is using the Yelp Reviews to rank the recommendations. … Nuture your Yelp account now. Claim it, and begin getting good reviews. Local search is a science, and you have to get that information out there.”
GigaOM: “Apple’s intent when it bought Siri was rumored to be building a search engine, though Jobs defused that speculation by saying, ‘We have no plans to go into the search business. We don’t care about it – other people do it well.’ But Jobs also said earlier last year: ‘On a mobile device, search is not where it’s at, not like on the desktop. They’re (consumers are) spending all their time on these apps – they’re using apps to get to data on the internet, not generalized search.‘ – With Siri, Apple doesn’t have to get into the search game if it can use Siri to direct people to the apps, services and information they need. That’s probably not a big money-gainer for Apple, but it could put a hurt on rival Google, which relies on search advertising.”
TUAW: “Curious about the iPhone 4S’s new voice assistant feature? So were we. – [We] tracked down a set of example phrases that the new Siri voice assistant is capable of understanding. It turns out that Siri can handle many categories of voice interaction. – Without further ado, here they are, ordered by interaction category, along with Apple-supplied examples of using each category.”
FC: “Don’t let her dulcet voice and easygoing, eager-to-please manner fool you. Behind Siri, the voice-controlled personal assistant app destined to power Apple’s iPhone 4S, lies the heart of a hardened combat veteran. That’s because the technology was spun out of the Defense Advanced Research Projects Agency (DARPA), the Pentagon’s high-tech research and development arm. … For now it can only respond to simple commands, but the technology underlying it is anything but. The problem with most speech recognition technology has been that it has a hell of a time with all-too human variations in speech – accents, dialects, intonation, enunciation, and slang. Tell it you want to hide under ‘a rock’ and it might tell you about ‘Iraq.’ Like the dream of the paperless office, which the advent of the personal computer was supposed to herald, speech recognition often makes more work than it saves. Siri promises to change all that, and you should thank the wizards at DARPA. While they didn’t create the technology, they incubated it. … I can’t wait to tell that to my Siri-powered iPhone, although I doubt it’ll know how to respond – not yet, anyhow.”
TC: “The most talked about element of … Apple event had to be Siri. The new feature of the iPhone 4S, born out of Apple’s purchase of the company by the same name in 2010, looks amazing. But one thing never mentioned during the keynote was a key piece of technology behind Siri: Nuance. – We first reported that Siri would be a key part of iOS 5 back in March. As we dug deeper, we learned that Apple and Nuance were involved in negotiations to make sure this could be a reality. You see, Siri does not work without Nuance. … So, is Nuance a part of Apple’s implementation of Siri as well? Yes. Though, don’t bother trying to get anyone to admit that. …Nuance is powering Siri. But Apple clearly struck a deal with Nuance which precludes them from talking about it. This is Apple technology, this is not about Nuance, is how I imagine Apple may put it. Apparently, Nuance is happy enough with Apple’s undoubtedly large check for this licensing agreement that they are willing to keep quiet.”
RWW: “Apple finally introduced the availability of the voice-command personal assistant app it paid $200m for today, called Siri. The military spin-off technology was both widely loved and often panned when it was available independently; it was either lovable Skynet or a fish on a bicycle, depending on who you ask. I tended towards thinking it the latter, myself. … But what do I want as a user – on my iPhone? I want Swype! Swype is a keyboard program available on almost every smartphone in the world except the iPhone. … It’s the fastest way to provide input on a mobile device. It’s fabulous and it’s incredible that Swype isn’t on iOS yet. I assume it’s because of Apple’s strict control over interface design and unwillingness to provide options in design. … Time will tell, but I don’t think Siri is going to be a killer app on the iPhone. Will it be used more than the current iPhone voice control? We’ll see.”
TUAW: “Since the iPhone 4S features the same A5 processor as the iPad 2, owners of Apple’s current-gen tablet have wondered if it’s possible that Siri, Apple’s new voice assistant, might be offered on the iPad 2. … Voice Control as it now exists on the iPhone 3GS and iPhone 4 doesn’t function on the iPad or iPad 2, but there’s a reason for that: the existing commands would be essentially useless on those devices. … On the other hand, Siri’s commands would be immensely useful on the iPad. … In fact, we’ve done some digging into Siri and found that most of the actual work of understanding voice commands gets offloaded to external servers. In essence, the iPhone 4S and its built-in processing functions determine what you said, while Apple’s servers translate that into what you meant and send that information back to your iPhone. … For the time being, Siri remains an iPhone 4S exclusive and one we have yet to test for ourselves. We look forward to putting this innovative feature under our interrogation lights once the iPhone 4S is released on October 14.”
Waxy: “In 1987, Apple released this concept video for Knowledge Navigator [the rest of the video is newer, probably circa 1996 or so, but the Knowledge Navigator part is from 1987], a voice-based assistant combined with a touchscreen tablet computer. … Based on the dates mentioned in the Knowledge Navigator video, it takes place on September 16, 2011. The date on the professor’s calendar is September 16, and he’s looking for a 2006 paper written ‘about five years ago,’ setting the year as 2011. – And … at the iPhone keynote, Apple announced Siri, a natural language-based voice assistant, would be built into iOS 5 and a core part of the new iPhone 4S. – So, 24 years ago, Apple predicted a complex natural-language voice assistant built into a touchscreen Apple device, and was less than a month off.“
katrce
hi siri
-
Gerrit Eicker
Well, she won’t hear you at this place. ;)
Baby
Hi siri
-
Siri
Hello
Yo Mama
Hi Siri
Adrianna
Sometimes she has a little attitude
Doll
Hey siri.
how you doing today.
bigL
Hi Siri
Gerrit Eicker
Computable Document Format (CDF)
Wolfram has released the Computable Document Format (CDF): bringing interactivity via computation; http://eicker.at/CDF
-
Gerrit Eicker
Wolfram: “Today we launched our Computable Document Format, or CDF, to bring documents to life with the power of computation. – CDF binds together and refines lots of technologies and ideas from our last 20+ years into a single standard—knowledge apps, symbolic documents, automation layering, and democratized computation, to name a few. – Disparate though these might appear, they come together in one coherent aim for CDF: connecting authors and readers much better than ever before. … With CDFs we’re broadening this communication pipe with computation-powered interactivity, expanding the document medium’s richness a good deal.”
RWW: “It isn’t simply readers who are meant to benefit from having more interactive publications. Wolfram says that the CDF is also designed to make it easier for authors and publishers to create and incorporate these knowledge apps into documents, arguing that up until now, these sorts of things have often required a knowledge of programming. CDFs can be created using the Mathematica software, and Wolfram insists that building a knowledge app is as easy as writing a macro in Excel.”
O’Reilly: “Wolfram’s tools create documents that can be shared on the Web, and are free for use by people who publish free documents. The tools can be licensed by organizations that charge for documents. Access to the tools can be on the Wolfram site (Software as a Service), or licensed and installed on your own server. – These tools look to me like a boon to educators, and I predict that all manner of publishers in the sciences and social sciences will license them. … Wolfram plans to release the format itself as what they call a ‘public standard.’ This is not the same as an open standard. … I assume Wolfram will keep strict control over the format, which draws a lot from the Mathematica language, and I doubt other companies will want to or be able to catch up to Wolfram in the sophistication of the tools they offer.”
Gerrit Eicker
If This Then That
ifttt, short for: if this then that, makes the Internet programmable, triggers actions to channels; http://eicker.at/ifttt
Gerrit Eicker
Digital Distraction vs. Utilisation
Growing up digital, wired for distraction; http://eicker.at/1m vs. open ideas about utilizing technology; http://eicker.at/1n
Gerrit Eicker
Link Building KPIs
Bencken: Tracking KPIs for link building is a critical step. 6 metrics you need to manage; http://j.mp/9At5tG
Gerrit Eicker
Summary: Getting Things Done (GTD)
Great book summary by Josh Kaufman for Getting Things Done (GTD) by David Allen; http://j.mp/dBOJfo (via @pherwarth)
Dan, and -
Gerrit Eicker
Kaufman: Here are 10 big ideas from David Allen’s Getting Things Done…
1. If your day-to-day life is out of control, it’s almost impossible to think strategically or plan effectively.
2. Define what being ‘done’ looks like.
3. Mental work has five distinct phases: Collect, Process, Organize, Do, and Review
4. Get everything out of your head.
5. Projects and tasks are two different things: track them separately.
6. Focus on the Next Action required to move forward.
7. Use the ‘2 Minute Rule’ for small tasks.
8. Use Reference and Someday/Maybe files for things that have no immediate next actions.
9. Build a trusted system that helps you keep track of your commitments.
10. Schedule non-negotiable time for a Weekly Review.Bonus tip: developing an effective personal productivity system takes time and experimentation.
Dan
For implementing GTD you can use this web-based application: GTDagenda.com
You can use it to manage your goals, projects and tasks, set next actions and contexts, use checklists, schedules and a calendar.
Comes with a mobile version too, and with an Android app.
-
Gerrit Eicker
Thanks for sharing. Added a link.
Gerrit Eicker 15:12 on 13. January 2012 Permalink |
Wikipedia: “In information technology, big data consists of datasets that grow so large that they become awkward to work with using on-hand database management tools. Difficulties include capture, storage, search, sharing, analytics, and visualizing. This trend continues because of the benefits of working with larger and larger datasets allowing analysts to ‘spot business trends, prevent diseases, combat crime.’ Though a moving target, current limits are on the order of terabytes, exabytes andzettabytes of data. Scientists regularly encounter this problem in meteorology, genomics, connectomics, complex physics simulations, biological and environmental research, Internet search, finance and business informatics. Data sets also grow in size because they are increasingly being gathered by ubiquitous information-sensing mobile devices, aerial sensory technologies (remote sensing), software logs, cameras, microphones, Radio-frequency identification readers, wireless sensor networks and so on. Every day, 2.5 quintillion bytes of data are created and 90% of the data in the world today was created within the past two years.”
ATD: “Over the last several years, there has been a massive surge of interest in Big Data Analytics and the groundbreaking opportunities it provides for enterprise information management and decision making. Big Data Analytics is no longer a specialized solution for cutting-edge technology companies – it is evolving into a viable, cost-effective way to store and analyze large volumes of data across many industries. … Big Data technologies like Apache Hadoop provide a framework for large-scale, distributed data storage and processing across clusters of hundreds or even thousands of networked computers. The overall goal is to provide a scalable solution for vast quantities of data … while maintaining reasonable processing times. … The barriers to entry for Big Data analytics are rapidly shrinking. Big Data cloud services like Amazon Elastic MapReduce and Microsoft’s Hadoop distribution for Windows Azure allow companies to spin up Big Data projects without upfront infrastructure costs and allow them to respond quickly to scale-out requirements. … To apply this new technology effectively, it is important to understand its role and when and how to integrate Big Data with the other components of the data warehouse environment. … Hadoop provides an adaptable and robust solution for storing large data volumes and aggregating and applying business rules for on-the-fly analysis that crosses boundaries of traditional ETL and ad-hoc analysis. It is also common for the results of Big Data processing jobs to be automated and loaded into the data warehouse for further transformation, integration and analysis. … Big Data adoption will continue to be driven by large and/or rapidly growing data being captured by automated and digitized business processes. … As we move into the age of Big Data, companies that are able to put this technology to work for them are likely to find significant revenue generating and cost savings opportunities that will differentiate them from their competitors and drive success well into the next decade.”
ORR: “Big data is data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the strictures of your database architectures. To gain value from this data, you must choose an alternative way to process it. .. The hot IT buzzword of 2012, big data has become viable as cost-effective approaches have emerged to tame the volume, velocity and variability of massive data. … The value of big data to an organization falls into two categories: analytical use, and enabling new products. … The emergence of big data into the enterprise brings with it a necessary counterpart: agility. Successfully exploiting the value in big data requires experimentation and exploration. Whether creating new products or looking for ways to gain competitive advantage, the job calls for curiosity and an entrepreneurial outlook. … If you could run that forecast taking into account 300 factors rather than 6, could you predict demand better? – This volume presents the most immediate challenge to conventional IT structures. … The importance of data’s velocity – the increasing rate at which data flows into an organization – has followed a similar pattern to that of volume. … A common theme in big data systems is that the source data is diverse, and doesn’t fall into neat relational structures. It could be text from social networks, image data, a raw feed directly from a sensor source. None of these things come ready for integration into an application. … The phenomenon of big data is closely tied to the emergence of data science, a discipline that combines math, programming and scientific instinct. Benefiting from big data means investing in teams with this skillset, and surrounding them with an organizational willingness to understand and use data for advantage. … If you pick a real business problem, such as how you can change your advertising strategy to increase spend per customer, it will guide your implementation. While big data work benefits from an enterprising spirit, it also benefits strongly from a concrete goal.”
Beye: “What is This Thing Called Big Data? – It’s difficult to avoid big data these days. More correctly, it’s difficult to avoid the phrase ‘big data.’ It has become such an integral part of the sales pitches of so many vendors and the blog posts of so many experts that one might be forced to conclude that big data is all-pervasive. The truth is far more complex. Even a definition of big data is elusive. … Big data is not, in essence, something entirely new. The problem is, to a large extent, one of scale; hence the name. However, the insights we currently have into these categories listed earlier and the different tools and approaches they require must be carried forward into how we handle these same data categories at a larger scale. … As a result, depending on your point of view, big data appears either as a giant wave of business opportunity or a huge precipice of potential technological and management pain.”
Beye: “What is the Importance and Value of Big Data? – Recognizing that big data has long been with us allows us to look at the historical value of big data, as well as current examples. This allows a wider sample of use cases, beyond the Internet giants who are currently leading the field in using big data. This leads us to the identification of value in two broad categories: pattern discovery and process invention. … Clearly, discovering a pattern in, for example, customer behavior may be very interesting, but the real value occurs when we put that discovery to use by changing something that reduces costs or increases sales. … More recently, combining data sets from multiple sources, both related and unrelated, with increasing emphasis on computer logs such as clickstreams and publicly available data sets has become popular. … The second approach to getting business value from big data involves using the data operationally to invent an entirely new process or substantially re-engineer an existing one. … For those contemplating investment in big data, the most important conclusion from this article is to recognize that there are very specific combinations of circumstances in which big data can drive real business value. Sometimes, of course, it is the price for simply staying in the game…“