cloud

How to Install Spark on Google Compute Engine

gce+sparkWhat is Google Compute Engine?

Compute Engine is an infrastructure as a service that lets you run your large-scale computing workloads on virtual machines hosted on Google’s infrastructure. Btw, if you wish to have a new machine under your arms in less than 5 minutes – It can be done in 5 easy steps.

What is Spark?

Apache Spark is an open source cluster computing system that aims to make data analytics fast — both fast to run and fast to write. To run programs faster, Spark offers a general execution model that can optimize arbitrary operator graphs, and supports in-memory computing, which lets it query data faster than disk-based engines like Hadoop.

So in order to enjoy from both worlds, we can  leverage the great options of large-scale cloud that Compute engine offer us and install Spark on it. Here are the few steps you will need to follow in order to do it.

Installation steps Continue reading

Standard
Chrome, HTML5, JavaScript, webdev

Google Cloud And Mobile Web

Screen Shot 2013-09-26 at 9.41.45 AM

Two talks in one day…

That’s what I did yesterday for Google Developer Group (GDG) Athens, Greece. It was a great opportunity to talk about the new cool aspects of google cloud platform (Yep, checkout things like: NodeJS on Compute Engine, App engine new support for technologies, Monte Carlo Simulations with App Script, Cloud storage, Big Query and many more). We talked for the first 25min on the new aspects of developing mobile web sites (and/or mobile web apps). In one word – go check out yeoman.io. Of course, there are many more aspects that are in the slides, so feel free to browse them and please let me know if I’ve missed something important that you are using in your mobile web project. Continue reading

Standard
Chrome, JavaScript, webdev

Install NodeJS On Compute Engine

NodeJS

In the past, I’ve showed how to run a NodeJS application in Compute Engine. It’s a good case where you wish to test ideas with App Script but when you move them to ‘production’ – NodeJS is the answer. After the Google Developer Live show we had at the end of August, I got few questions on how to install NodeJS inside Compute Engine instance.

Here are the (easy) steps I did before the show in order to save us time: Continue reading

Standard
HTML5, JavaScript

Google Cloud Platform – App Engine and Beyond

Screen Shot 2013-07-11 at 2.43.33 PMFew minutes ago we finish a great event in Campus TLV. We had the pleasure to have Dr Peter Magnusson (Director of Engineering for App Engine at Google) and Greg DeMichillie (Director Product Management, Cloud Platform at Google). For the ones, that do not know what is GAE: Google App Engine is a complete development stack that uses familiar technologies to build and host applications on the same infrastructure used at Google. It is used by some of the world most successful and demanding global applications such as SnapChat and Khan Academy. We also cover Google Compute Engine which allows everyone to Run large-scale computing workloads on Linux virtual machines hosted on Google’s infrastructure. I’ve spoke about Google Cloud Endpoint and you can watch the full talk on my project’s site.

Ido talking in the campus on cloud

This is the raw recording form the event of all three talks. I hope to have an edited version soon…

app_engine-256

Psst… For the next Cloud meetup – Please sign for IGTCloud group.

Standard
Chrome, JavaScript

Monte Carlo Simulation On Compute Engine/App Script

App script on Ido's project place

This week in Google Developer Live Israel, we show you how to work with Google App script to run Monte Carlo simulations and get the results in an excellent informative way inside Google spreadsheets. We took it further and explained how to run these scripts on your instance machine inside Compute Engine with nodeJS.

This is a powerful option because the ability to ‘try’ quickly on App Script and then take the code to ‘production’ (=more efficient way) on Compute engine gives you more productivity.

So what are we doing?

First, let’s touch on the definition. This problem-solving technique approximates the probability of specific outcomes by running multiple trial runs, called simulations, using random input variables. The remarkable aspect is that we are steering the scenarios by using randomness.The randomness helps us ‘try’ each future strategy in a way that includes our probability per variable. This allows us to fine-tune our trial runs to answer a question with several variables. It will help us find the best way to spend time with our salesperson.

 Monte Carlo Simulations Steps

1. Define a model with a domain of possible inputs.

2. Generate input values randomly, sampling from a probability distribution over the domain.

3. Perform a deterministic computation on the inputs.

4. Aggregate the results. Continue reading

Standard
webdev

Compute Engine In 5min

Screen Shot 2013-07-11 at 2.43.33 PMThis week, we showed in GDL-IL (Google developer live Israel) how to work with google compute engine. Google Compute Engine offers scalable and flexible virtual machine computing capabilities in the cloud. You can solve large scale processing and analytic problems on Google’s computing, storage, and networking infrastructure. It’s also great option to ‘play’ with a server (web service) in the cloud and scale it efficiently. It’s *open for all* and it got some powerful features like: Continue reading

Standard