Decentralization


At the moment there is a huge debate raging in the Bitcoin community on whether or not there should be a hard fork into two different currencies along the same lines as what happened to Ethereum in 2016. As can be imagined, the debate is intense with many folks taking extreme sides. My goal for this essay is not to weigh in on that battle as my technical understanding of Bitcoin is limited in this debate compared to many of the developers and other folks who are staking their businesses and focus on it. But there is another idea that is constantly springing up in many of these debates and that is this idea of decentralization and more importantly, building decentralized applications. So why is the concept of building decentralized applications gaining prominence?

First, there are three types of ways that applications can be built: centralized, decentralized, and distributed. The majority of general applications being built are centralized, that is, there is a unique core node that must be used in order to access what the app is offering (be it data, an API, or your account) and the core node instructs all of the connected nodes as to what to do. If we take a step back and analyze this idea we can see that all information being produced will flow through a single center (or node). Every person who uses these services is dependent directly on this central authority maintaining the power to send and receive information. Google, LinkedIn, Facebook, and Amazon are all built on centralized stacks and this design works powerfully for them both technologically and business wise. 

Then there are distributed and decentralized applications. A distributed system means that computation is spread across a network of multiple nodes which helps speed up computing and latency of data access. A company like Google builds distributed software to help speed up their services. A decentralized application means that there is no central node that instructs the other nodes on what to do. Bitcoin is the ultimate decentralized application because if one node fails, it will not have an effect on any of the other nodes and the network will continue to operate. For the purposes of this essay, I am going to skip going through anymore detail on distributed systems. 

So why should care about decentralized systems/applications especially since centralized systems already work so well for these companies already? For one, there is a rather large possibility that these applications will be used for their superior incentive structure, resiliency, transparency, and distributed nature. Using a blockchain (peer-to-peer distributed ledger) to a form a trustless system, value can be created using cryptographic tokens, which can then be used to access the application. As I stated in the previous paragraph, the premier decentralized app is Bitcoin which simplifies the traditional financial system. In order to access the network, one must own some bitcoin, which can then be used to store value, or easily transfer it from one wallet address to another. For example, cross-border payments are made easily since the value isn't being transferred through several financial middleman.

Another way that decentralized applications are being built is as protocols that use another blockchain, such as Ethereum, and issue their own tokens to function. One example interesting example is the Golem Project; it lets users access another users computer using their tokens as the exchange of value. For example, if I set up some spare computers and put them on the Golem network, anyone with Golem tokens can use my computers in exchange for those tokens. We suddenly have a way to put our spare CPU's to work. This has been an idea I have thought about considerably using bitcoin as the exchange of value instead. Either way would work well and put spare computers to work.

But let's not get ahead of ourselves quite yet. Centralized services still absolutely dominate the vast amount of users and will continue to do so over the coming years. It may not even be until innovation begins to slow down or companies begin having other problems that will eat up their time (this could be anything from internal problems to governments/states coming down hard on them). If this does happen, then decentralized systems built on blockchains could start becoming more well known as easier and stable computing platforms. In fact, I am thoroughly convinced that these systems will be incredibly important for businesses, customers, and citizens. But it will be a long road before we get there successfully and my goal is help pave it along the way.

HTTP vs HTTPS


The Basics

HTTP

Security is becoming increasingly important as better online experiences now involve trusted third parties and good encryption. A basic understanding of how this works is knowing the difference between HTTP and HTTPS.

Hypertext Transfer Protocol (HTTP) is the system used for sending and receiving information across the internet. It's what is known as an "application layer protocol" so its main focus is on how information is presented to the user. This option doesn't care how data gets from point A to point B and it is also "stateless" which means that it doesn't remember anything about the previous web session. There is a benefit to being stateless which is that there is less data to send meaning there is increased speed. 

The most common use for HTTP is to access HTML pages, which are the backbone of the websites we visit on the internet. However, it is important to remember that other resources can be accessed and utilized through HTTP as well. In fact, this is the most common way that websites that do not house confidential information (such as credit cards and/or usernames and passwords) are setup.

HTTPS

Secure Hypertext Transfer Protocol (HTTPS) is for all intents and purposes, a similar system used for sending and receiving information across the the internet, it's just the secure version. The protocol was developed to allow for secure authorization and transactions. We don't want malicious actors gaining access to the private information we are creating and HTTPS adds an extra layer of security to that exchange of confidential information. That extra layer is made possible because it uses a Secure Socket Layer/Transport Layer Security (SSL/TLS) to move data back and forth. Neither protocol cares how the data gets to its destination although HTTP cares about what the data looks like whereas HTTPS does not.

Google actually prefers websites are encrypted with HTTPS because of that guarantee of extra security. When a business owner, developer, or webmaster goes through the motions of obtaining a certificate, the issuer then becomes a trusted third party. The information in the certificate is used to verify that site is what it claims to be and finally the user/customer that knows the difference between HTTP and HTTPS can by buy with confidence, giving electronic commerce more credibility. For anyone maintaining a site with heavy traffic, Google and the other search engines will put priority on sites with security and keep them boosted in the rankings as long as the multitude of other SEO related work follows their guidelines.

More Detail

Data sent using HTTPS is secured using via the Transport Layer Socket protocol (TLS) which provides three important layers of protection: 

  1. Data Integrity - Data that cannot be modified or corrupted during transfer without being detected.
  2. Encryption - Encrypting the exchange data to keep it secure.
  3. Authentication - Proves that the sites users/customers communicate with the intended site.

These three layers are the main motivation behind the HTTPS protocol and help prevent against eaves dropping and tampering with the communicated content via man-in-the-middle (MITM) attacks. 

How do browsers know who to trust?

Browsers come pre-installed with certificate authorities, meaning they know who to trust. Likewise, the browser software is trusting those authorities will provide valid certificates. A user/customer should be able to trust an HTTPS connection provided the following are all true:

  • Trust that the browser software correctly implements HTTPS with the correct pre-installed certificates.
  • Trust that the certificate authority will vouch only for legitimate websites.
  • The website provides a valid certificate signed by a trusted authority.
  • The certificate correctly identifies the website.
  • The user/customer trusts the protocols encryption layer (SSL/TLS) is secure against eavesdroppers.

It is becoming increasingly important to use HTTPs over insecure networks such as public WIFI since anyone one the same local network can discover sensitive information using packet sniffing. The same goes for using WLAN networks which can engage in packet injection to serve their own ads on webpages. Doing this can be exploited in many ways such as injecting malware onto those webpages to steal users' data and private information.

The case for using HTTPS on your own websites

With each day it seems we learn that more and more information about global mass surveillance and data being stolen by malicious actors. Because of this, the strongest case to use HTTPS is that you are making your website more secure. There are however limits to using HTTPS as it is not 100% secure. It will not prevent your website from getting hacked or stop phishing emails getting sent either. It's importance is in the fact that if you have users/customers that are logging in with sensitive information (such as passwords, social security, etc.), then setting up HTTPS is the absolute minimum price and precaution that should be taken in order to protect them. And with security, you will build trust.

The Blockchain Meets Seattle

Earlier this week I had the opportunity to be invited to a private event here in Seattle regarding Blockchain technology. The even itself took place on he 48th floor in the old Washington Mutual Tower. The fog disrupted what might have been a beautiful view. The event itself went by rather quickly but it did a great job of explaining how the Blockchain technology can be leveraged in more ways than just creating the killer app "Bitcoin". Some interesting points that were brought up were:

  • Using the Blockchain as a ledger to keep track of IP, land titles, and other assets
  • Goldman Sachs has filed a patent for crypto-security settlement on a blockchain.
  • Microsoft has declared that the Blockchain is one of the "key must win workloads" for their Azure cloud platform and business. They are also collaborating with major U.S. banks using the technology. 
  • Some governments are already beginning to invest in their own local, or private, blockchains.

What was most surprising to me was the actual plans and partners Microsoft has for their Blockchain-as-a-service (BaaS) tech they have on their Azure cloud platform. Some of those partners include:

  • Bitpay
  • Multichain
  • OpenChain
  • Coinprism
  • Augur
  • Slock.it
  • Ripple

Putting tech like this in the hands of developers easily will be a huge factor in spreading it far and wide. The event was exciting to me as I was able to speak to a number of people about an area in tech that I have been involved with for a few years now and it also showed how many people are interested in learning more. I hope to be a part of those bringing this technology to the masses over the upcoming years. 

Yesterday

"It's no use going back to yesterday, because I was a different person then."

- Lewis CarrolI


Life has changed quite a bit for me and I am ready to get back in to writing again. I miss it terribly and have made the decision to start posting some of the ideas rattling through my mind as of late. Writing really is the best way for me to clarify some of my thinking.

Looking Forward

As is standard practice in the New Year, many people enjoy writing about what they think will happen in the coming year ahead. It’s not something I’ve ever done before and so instead of just quickly throwing together a quick list and publishing it, I thought it would be better to look at a few important developments taking place in the world and see where they may be heading. So here are my thoughts on the four spaces that could have an impact on the global economy.

  • Emerging markets are going to go through quite a bit of pain due to the price of many commodities falling to earth. China seems to be going through a bit of turmoil within its economy which can have a negative ripple effect across the global economy as a whole. The country is beginning to really experience a huge amount of pain brought on from the large debt that has been built up in some of its industries. For example, since the price of steel and iron ore have fallen so much since early 2015, the construction boom has slowed down and claimed many jobs in Europe. In response, fiscal and monetary stimulus will be used as best it can in order to help bring back demand. Doing this could hinder the exchange rate for yuan which is already feeling a large amount of pressure from the outflow of capital. If the yuan is benchmarked against a basket of currencies, its value could decline which could lead to a serious erosion of value in market valuations in much of Asia. What should be watched for is just how much China’s economy and market movements effect the rest of the worlds markets. As China grows bigger, the countries presence will begin to be felt in countries that it has invested in or have invested in it, for better or for worse.

  • Bitcoins presence will also begin to be felt in both currency markets and some products and services in the financial industry. It is the easiest way to store and send monetary value using the internet and the bigger it gets, the more people will learn about it. Many venture capitalists and industry players still talk about finding that killer app that will bring it further into the mainstream, but it still may be slow going on that front. Many banks and financial institutions including Visa and Goldman Sachs are starting to use the underlying technology, called the Blockchain, with the help of many fintech startups that have built a product or service that these companies will find useful. The use of technology by these companies will help push it towards a larger consumer base to use. There should also be more massive market movements expected for two reasons: 1) the community is still divided over the block size and (with a major core developer leaving rather loudly) and 2) its presence has been felt in China with the price popping when there are problems with the market and traders reacting by quickly moving resources in Bitcoin. Big things could make some headway this year provided there are some creative uses of the technology and the community does not remain split.

  • Security and privacy will remain on everybody’s radar as important subjects to understand what is going in. Security, already a huge geopolitical issue, will become a subject that effects more and more people as technology rapidly advances. We walk around with devices that can be tracked at all seconds of the day so it should be fully expected that more security related services and products will make their way to the market. Nobody wants their phone hacked into since it carries such a great amount of details about our lives and there will be many companies that answer that concern. But the other side of that coin is the subject of privacy; a slippery slope due to so many differing opinions of how to define privacy. Right now, most people don’t understand cryptography or the mathematics behind what makes it work; nor do they know to what extent companies and states can track their every move and search query and place all that information into a large context about our lives. On the business side, in the case of retail and advertising, I side with giving the consumer the choice of whether to do or don’t want to be tracked and sold to. If they do, to what level of privacy are they willing to forgo their privacy? And in the case that consumers aren’t given that choice, tools should be made available for anyone to use to give them the privacy they expect. And in the case of security, companies and states should work together to come to a middle ground that gives both what they need. This being the year of a presidential election, it should be expected to hear this subject brought up quite a bit.

  • Lastly, Artificial Intelligence (or machine intelligence) will become a bigger part of our lives; probably even in ways that we aren’t fully expecting to happen. But where it will remain most important to us will be in our working lives. As more and more rote work is able to be automated easily by computers, we will find them working by our side and helping us achieve things we didn’t thing possible to an even greater degree than they already are. The use of machine learning will help with completing sets of tasks and jobs more efficiently. Large tech companies like Google, Facebook, Baidu, Alibaba, Amazon, and Microsoft will help pave the way to a greater use of AI by making their products more intelligent and useful. These same companies will also go a step further (some already have) by open sourcing their code so that more researchers, smaller companies, and independent developers will have access to the same tool sets, thereby producing new products and creating a healthy ecosystem. It can even be thought that these same smaller companies or products will be purchased by the larger players and seamlessly integrated since the underlying technology will be the same. It’s exciting to think about what kinds of new products can be made.

These four developments will have a large impact on the global economy and marketplace whether positive (AI related products, Bitcoin, Privacy) or negative (Commodity prices crashing, Security) and it’s interesting to think about what lays ahead for us. There are a few developments I left out that will definitely have a large impact on the world and those include the price of oil ($28), augmented and virtual reality, and a U.S. Presidential race. Should be an interesting year that lays ahead.

The Prediction Business

Businesses are in the prediction and risk business, a bold statement if there ever was one. No matter what, when running a business, there are going to be things that are known and things that are unknown. We can control the things that are known to us and act on them in clear ways. But the unknowns are a bit trickier because they are unknown. We just don’t know what they are there. For example we don’t know how our customers or the market are going to respond to a new product. Or if we should be doing a heavy amount of fundraising based on the customer growth curve and whether or not we have sustainable operations to continue heading down this path. This is the risky part of the business.

In order to better understand how to predict these variables, more companies should be collecting as much data as they can about their businesses and understand it the best they can. Data is the 21st century oil. Reading it is the 21st century literacy. Big Data is what the press is calling it. Almost all companies collect data now and because of this, it is much better to understand how it works, what is the information is trying to convey, and how the information can be used to make a forecast, tell a story, or just inform. General statistics has always been used to mine and learn from data in the past. But with faster computer speeds and access to an almost unlimited bandwidth, it is now much easier to run much more advanced algorithms over the data using what is known as machine learning. What is machine learning? It is applying statistical models to the data you or your company has so that smarter predictions can be made about the data you don’t have.

With each new set of data we encounter, new uses for algorithms must be found. And while machine learning has a great amount of potential to the way companies approach their business problems and the way entire industries operate, it can still be thought of as a branch of statistics that is to be used on big data. And the tools that machine learning bring are designed to make better use of that data.

How can we approach thinking about big data and machine learning along with it? The enormous scale of data available to firms can be challenging; using machine learning is as much about data analysis as it is about adapting to the sheer size of any particular data set. A great way to think about data is how long and wide it is. What is meant by this statement is our data set will be long depending on how many rows it happens to have. Let’s say we are analyzing a large company’s data and what it will look like; we can imagine each row being one unique customer. Depending on the size of the company there could be up to millions or even billions of customers. So with that line of thought, the more customers there are, the longer or higher our data set will be. Width then corresponds to the number of columns in the data set. So in our case, each column is considered a unique variable assigned to our customers. For example, our columns can be purchase and browser history, mouse clicks, and even text. This data set can become rather large and overbearing and this is where machine learning makes use of a tool set to better analyze wide data.

We can further refine our initial question down further by asking what machine learning is used for? The most common application is to make predictions and this is why it is becoming so important to businesses. Being able to make predictions about data that isn’t available can be used to formulate sales, marketing, operations, and financial strategy. Here are a few examples of how it is used in industry today:

  • Personalized recommendations for each customer. (Amazon product recommendations, Spotify and Pandora recommending new music, and Netflix movie recommendations)
  • Forecasting customer loyalty (How often they shop with a company down to the time and what they consistently spend their money on.)
  •  Fraud detection and credit card risk (More banks and insurance companies are using their data to make predictions about what customers may be a moral hazard)
  • Facial recognition software (Facebook makes great use of this when it recommends who should be tagged in a photo)
  •  Advertisements that create their own copy and images (M&C Saatchi partnered with Clear Channel UK and company called Postercope to create these ads.)
  • Personalized assistants (Apple’s Siri, Google’s Now, and Microsoft’s Cortana are just the big name examples of what can be accomplished. There will be many, better assistants down the road.

The common identifier is the need for a unique business process and the decision that must be acted upon to get to that accurate prediction. Each of these examples come from complex environments where a correct decision depends on many different variables. (Our wide data). And each prediction will ultimately lead to an outcome with whatever it is helping the model become continuously better.The business value of machine learning is enormous even with its limitations are taken into consideration. It is focused on prediction which means the model of the environment might be all that is needed to make the right decision.

So let’s get into how machine learning can be used in practice. Within each machine learning algorithm there are generally three broad concepts. They are:

  • Feature Extraction: This determines what data to use in the model.
  • Regularization: Used to determine how the data are weighted.
  • Cross-Validation: Tests the accuracy of the model.

What each of these concepts does is separate the “signal” from the “noise” which is common in most every data set and helps sort through the mix to get to better predictions.

Feature extraction is the process where the variables that the model will use are discovered. There are times where all features are dumped in to a model and used but more often than not this doesn’t happen due to overfitting. Features help aggregate important signals that are spread out over the data. For example, if your company runs an online music store, each feature could correspond to musical genre, record label, or even the artist’s home county. Once these data points are collected they are combined through automation that clusters the features together and the model can then analyze customer predictions. A very well-known business case is Netflix’s movie recommendation algorithm. The more each customer uses their product, the more data points they are able to collect about that user and the company is better able to predict what movie or television show the customer is interested in watching.

After we have our features chosen we must understand if the data we have been collection and what it is being combined reflects a signal or noise. So we begin by playing it safe with the model using regularization. This is a way to split the difference between a flexible model and a conservative model. For example, one effect is known as “selection” which happens when the models algorithm focuses on a smaller number of features that contain the best signal, discarding all other features. Regularization helps the model stay away from overfitting, (overfitting is when a model learns patterns from the data that ultimately are not helpful and won’t hold up in future cases) and helps it learn from both signal and noise.

In order to test the accuracy of the models predictions, a process is used called cross-validation. To test that the model is “out-of-sample”, which is when predictions are made on data we don’t have based on data we do have, our initial definition of machine learning. This is done by splitting the data into two sets called the training and test data. The model is first built using the training data and then more tests are done with the use of the test data. Keeping a clear partition between the two sets is instrumental in not over estimating how good the model actually is.  

There are many examples of machine learning being used in production that we use on a daily basis. In some cases, we might not even be aware that our technology is using it in the background. Netflix was used as an example of a business that makes great use of its data. Amazon is also extremely data driven with their product recommendations being used skillfully with each customer who shops with them. However, the company that probably uses machine learning the most right now is Alphabet, Inc. (The Company formerly known as Google.) Machine learning not only guides how their search engine works so efficiently, but is also used in Google Translate, Nest, their self-driving cars, Google Now, and many other products they offer. The more data they collect from us, the better they will be able to fine tune their algorithms in their products so that they will interact with us seamlessly.

One final intriguing example is how a digital agency is using artificial intelligence to create ‘self-writing’ campaigns in London. How it works is the ad itself is placed on a bus stop and has a camera connected to it. This camera registers commuters’ engagement based on whether they look happy, sad, or neutral. Then, an algorithm executes various responses based on the commuters’ responses to the ad. This campaign in particular only used a fake coffee brand since it was more of a test than anything. But if the proof of concept works, we may start seeing more interactive billboards out and about.

Being able to make the correct forecast and predictions for your company isn’t something that can be done with 100% accuracy, but businesses that do utilize the data they are collecting to its fullest potential find they are better able to cope with the uncertainty of variables they can control. Forecasting isn’t about getting the answer to your questions correct, because that isn’t going to happen. Forecasting is about being able to make sound judgement from the data and the algorithms used to mine that data will help anybody or business get a bit closer to the answers they are looking to find.

The Creation and Capture of Value

In order for a business to create value after cost it must create and distribute that value in the most efficient way possible. It can be considered the starting point for any and all businesses which leads to our first question to think about: how is that value created? Simply put, it is created through work. That work could be anything from administrative tasks (such as filling out the right paper work for customer orders), technical (deploying code to servers), and creative (marketing copy, product and/or logo design, etc.). The business can then create value through that work, sell or trade it to a customer base, and capture some of that value through profit. Based on this definition, we can begin to clearly see that businesses add value in more ways than just making a product and selling it. Every moving piece of that business should be moving towards the end goal of creating and capturing that value.

Now let’s start thinking about the types of value that can be created. First, let’s remember that not all types of value are created equal by any means. A value that is considered a commodity is easily replaceable in the minds of customers. For example, if your products aren’t distinguishable from your competitors, then that competitor will be primed to take your place should your business falter by any means. There are ways around this though; if your company is able to create a new and more efficient process for doing business or in possession of unique skills focused on the creation of value, then you will be able to more readily differentiate from those trying to eat your lunch. Having any of these things is a competitive advantage and should be fully utilized towards the goal of value creation.

Measuring value creation is important if only so that you understand what the value is that your business is making. The first method is by measuring revenue. Revenue tells you that the way your business creates value was worthwhile to your customer base since they are willing to pay for it. Notice how I didn’t say profit. Many businesses successfully create revenues but no profits (think Amazon), but many will not be able to do this for very long. A business needs a profit in order to survive and sooner or later a lack of making any profits will bring that business down, no matter how great the product or service. (Amazon has been able to successfully navigate this pasture by shrewdly reinvesting its revenues into future initiatives such as AWS.)

Then there is perceived and exchange value, which are interrelated. The exchange value is straightforward in that it is the amount of value exchanged between a buyer and seller for a product or service. For example, if you go to the store and purchase a pair of shoes, the price you pay for those shoes is the exchange value.  The perceived value is defined by our perceptions of usefulness of the product or service. In economics there is a consumer surplus (C.S.); when the consumer surplus is greater than zero (C.S. > 0), then the customer is better off making a purchase than not. So value is created when the perceived value of a product or service has a certain degree of usefulness, consumer surplus is greater than zero (C.S. > 0) and that same value is then exchanged to the seller.  

As we can see, it’s incredibly important for a business to create value. It won’t survive for very long if it doesn’t create value by differentiating itself in the marketplace or not making a profit in the long run. In order for a business to survive, it must capture a portion of that value it is creating. If businesses ultimately want to succeed, they must think clearly about how they are going to capture the value they are creating. Businesses that don’t do this may be leaving money on the table.

There are a number of different ways to capture value with some being more common than others. For instance, price based on value changes according to the offerings worth to the customer. What this means is businesses don’t set their prices based on what their competitors are doing. They might also discontinue the process of marking up prices based on production costs. Instead, what they are doing is looking at what their customers want and setting their prices accordingly.  What is important is that the customers’ perception of value must be discovered. This is obviously different with each customer but there are different models of discovering this missing piece. An obvious example is auctioning, which doesn’t work with all business arrangements but is incredibly effective when it does. The most common example of an auction is with online advertising, where each buyer sets the price and the seller can choose whether or not to take that offer. Of course this is controlled less and less by human interaction and is guided more efficient using algorithms to guide the software. There are some downsides such as prices that may be less than satisfactory for the seller, but must be honored regardless.

Another model is known as demand-driven pricing; what this does is let the price change due to the fluctuations in demand for a product or service. The most common example to date is Uber and it’s constantly changing prices. Although there can be a large amount of complaints if the price is too high, there are reasons for it, and Uber acts on those reasons with a mechanical efficiency. The business is a money making machine. How it works is the company raises and lowers its prices based on demand for rides. Other factors that are taken could be day of the week, whether it is a holiday or not, weather, and even city, etc. Using these variables (and many, many others), the company can maximize its profitability by knowing how many cars to have on the road and in what location at any given time. Demand-driven pricing at its finest.

A further form of value capture model enables customers to set their own prices. Although this doesn’t take place in most industries, it is especially prevalent in the travel industry where buyers can decide what price they want to pay and sellers can take it or leave it. Unlike auctions though, this transaction is mostly kept between the buyers and sellers. Doing this lets the seller maintain their prices even if they are discounting for incremental sales.

Next up is for companies to capture value by using two-sided market forces to their advantage. Although the name may not ring a bell to most people, it is a model everyone has seen in action and is used efficiently by media companies. For example, many publications are free for the general public to take (think local periodicals in many major cities) and they make their money, in turn, by charging advertisers to place ads in their publications. The money is then used to subsidize the content that is created for the publications and the process is continued. Although Vice Media is a much larger entity now, when it was just a magazine, it was free to anyone (who could find it) and the vast amount of value was captured by charging advertisers (especially American Apparel). Of course they made money with subscriptions, but most people just searched high and low for a free copy.

Then there are those businesses that use what’s known as the “price carrier” in their offerings. This is the experience that businesses will hang a price tag on while customers may not be coming through the doors just for that in the first place; they may be coming for something else entirely. Think about it, we will sometimes purchase a product or service to gain access to something else the company is offering, but has no price tag on. Starbucks is a great example of this. The majority of their customers walk through their doors for a quick coffee before work in morning, but there are those people who come in just for the wi-fi. Using it isn’t for sale, but purchasing a coffee or pastry will give you access to it. So a good question to ask yourself is whether or not your business is hanging the price tag in the right place. What would happen if you moved it to something else?

One of the most prominent examples of value capture is called the razor-and-blades model, which was pioneered by Gillette. It’s a rather simple model: customers get the razor handle for free, but must purchase blades continuously since they get dull rather easily. And charging for the blades is where the value capture happens. This model was also utilized by technology companies who sold printers and printer ink replacement cartridges. The printers are always cheaply priced; the ink is not.

The final model to talk about is one we all know well since the phone carriers are incredibly efficient at using it; it is known as bundling. How it works is the price of the new phone is subsidized by any extra hardware, software, and data features we purchase with it. The phone is cheap; the options that are bundled with the purchase are where the value capture takes place. Car dealerships also excel at this too; when you go to purchase a car, the sales people will usually bundle in many products or services you may or may not even need since those add-ons are where the money is then made.

Both value creation and value capture are incredibly important to keep your business running smoothly over a long period of time; more so than your competitors. Both are equally important; both need to be studied rigorously for the best understanding of how the complement one another. Value creation is the work that a business needs in order to create value to offer to their current and potential customer base; capturing that value is what will keep customers happy and the business running smoothly. Combining the two will be what ultimately either keeps a business alive. And that is something to constantly been thinking about when focusing on your own business.

Broken Windows

Technology is changing a large aspect of how we live our daily lives without giving us the time to slow down and catch our collective breath. This aspect of technological change should be enough to make anyone pause, but there doesn't seem to be any end in sight. This isn't a negative, it's just what it is at the moment. I'm also not saying that this is a good or a bad thing; it just is. Where my concerns are starting to focus on is what the trade-offs we may not be paying attention to are. Technological change is a Faustian bargain and it seems to be happening before our eyes without an consideration to the trade-offs of living our lives around it.

What am I talking about regarding trade-offs? Technology gives and technology takes; there is always a hidden price to pay for letting it become the centerpiece of our lives. This isn't at all obvious to most people. For most, the price they pay may be of greater importance than the trade-off. For example, take Facebook. In order to use the service we must sign up and agree to the terms before we can begin using it. That agreement is the price right there; we are agreeing to let Facebook do whatever they want with the data that they collect about us. And there are a variety of resources they use that data for such as marketing and advertising, making their product stickier to customers, and user experiments. The ultimate price we pay is our online privacy. I'm not condemning or condoning it; again, it's just what it is.

Online services aren't the only technologies with major trade-offs. Another example is smart-phones and their now ubiquitous nature. They are fantastic devices that many people in the world use on a daily basis, but at what price? How difficult is it to have a conversation with someone without them staring at their phone in the middle of you talking with them? How has it changed our social interactions with other people, and is that a good thing? Automobiles are another example; we were suddenly able to get somewhere faster and more conveniently than ever before and that revolutionized the start of the 20th century due to not only how they were made (Ford lines) but to how it gave people the ability to travel easily. Now we know that they are responsible for contributing to air pollution and traffic gridlock. They have done an incredible amount of damage to our environment and that will not be easy to fix, if it ever can be.

Technological change is our modern day version of Bastiat's Broken Window Fallacy. We buy into a new technology and can only see one side of the benefits. For example, we might start using an app that hires people to deliver us goods (such as food or clothes) and/or services (such as taxis). By doing so, we revel in the new found freedom from the convenience it provides our lives and the time it now affords us, allowing anyone to now focus on much more important parts of their lives. But this is only one side of the story. We aren't opening our eyes or even looking for the trade-offs that are taking place. And this needs to change.

Very rarely do we ask what this technology is undoing for us. What is the other side to that trade-off? How does using this new technology effect the people who are working for it (such as drivers for Uber or deliveries from Postmates). Are the laws being changed to accommodate these new companies and should they even be changed in the first place? What will be the long term effects of those changes to our economy and even our own jobs? There are some very interesting critiques of both Postmates and Uber.

These are all questions that are difficult to answer but the very real point that I am arguing about in this essay is that most of us know very little about the social and psychological effects that new technologies have on our society. And it's in our best interest to really start thinking about, and understanding the costs that these technologies bring to us. 

Predicting through Randomness

I wrote this back in January of 2015. I wanted to wait until closer to the end of the tournament until I posted it. What I found interesting about it was that although I didn't answer all of the questions, those questions I did answer gave me a fairly low brier score, which is used to show how accurate the forecast happened to be. Sticking with financial markets and indexes seems to suit me. Political maneuvers we're a bit harder to tackle. In the end though, forecasting is a great tool to have in the toolkit, but it should only be used by those who are going to focus their entire attention span on the subject.

7/3/15 - Just received my feedback report and I placed in the top 5-6% of the tournament. Will have to keep focused on this. 


Forecasting is hard. There is no way around it. When you are trying to forecast the outcome of a certain idea or situation there is so much back and forth information that is super hard to do. There are people who make entire careers out of forecasting, some more on force of personality than actual hard analysis, and some on actual deep dive analysis. I've been thinking about forecasting now for the past few months since I was accepted into the Good Judgement Project tournament. And all it once it is a fantastic way to learn of what is happening in the world around us and quickly become humbled by what we know and think we know. I've been learning quite a few lessons since I've joined including being humbled by what is really nothing more than my opinion on something. Forecasting, if done correctly, forces you to look hard at the reality of what is happening and make judgments from there. And many times, we have to redo what we were thinking about. 

For a quick history lesson, the Good Judgement Project was started in 2011 by Philip Tetlock and funding from IARPA. Over the course of the next few years, the researchers found that accurate training plus a strong dose of statistics, pyschology, game theory, and interactions among team members helped create highly accurate forecasters. These people were not government trained at all. Most of them were only getting their news from online sources. The same same sources that most anyone has access too. However, these amateur forecasters were displaying a 30% more accurate prediction than those in the intelligence community with access to classified information. I thought this was pretty exciting and applied quickly thinking that it would be great way to finally apply some of the game and probability theory I learned in college. But that wasn't the only reason I wanted to join. 

Thanks to the influx of more technology driven businesses there has been larger amounts of data available not only on open source, but for companies as well. Most companies have been using data to understand their customers for years (the credit card companies can paint an accurate portrait of a consumer and do it with ease), but there are new techniques being used to accurately mine that data. These techniques come from a brand of artificial intelligence and are focused mainly in an area called machine learning. What machine learning does is construct algorithms, study the results of those algorithms, and learn from that data. For example, there is one algorithm called "Association Learning or the Apriori Algorithm". It is generally known as the first algorithm data miners try. What it does is learn interesting relationships among data in large sets. An easy example would be the groceries you purchase from the store. Management can mine each transaction that is made and discover if there is any correlation between items being purchased. Perhaps people who buy apples also buy cheese. Management can then market their products in order to keep that relationship continuing and perhaps even discover more relationships. Many companies that do this can learn quite a bit about their customers (even though some argue this can't be done right now, with the way technology is growing, it probably will soon.)

For me, gaining more confidence in my ability to apply mathematical probabilities to real life events is something I've been wanting to do but never had the proper outlet to succeed at doing. And having spent the past five years learning all I can about computer programming, this finally feels like an excellent outlet for me to pursue. And sure enough, once I started learning how to apply data mining techniques to large datasets, hours started flying by as I immersed myself into this new world. Gaining the right information to then apply as knowledge is becoming the new currency in the world today. There is so much raw data that anyone who can sift through it and correctly analyze and accurately communicate what it means will easily by a one-eyed king in the kingdom of the blind. 

There are a few rules and although I'm not going to give a rundown of the entire training materials, I will explain a few ways it is currently being used.

  • It is a foolish to ask for predictions for the fundamentally unpredictable. What I mean by this is that although there is a method and process for accurately forecasting the probability of an event happening, there are still mostly events that cannot be predicted. The best we can do is ask ourselves the right questions and move forward from there. But the big key is knowing the right questions to ask and when to ask them. Trying to predict where the ball will land in a game of roulette on every play is mathematically impossible (unless there is cheating involved because the probability of guessing correctly just three times in a row is .00182%). However, if you know what the odds are against you (trust me, the house has a 5.26% advantage on every spin) and what your long term expected value for continual play is, then game becomes much more manageable. The movement of the financial markets could also be thought of as unpredictable and if you are going to play that game, your best bet is to have a strategy and system in place. (I'm not going to go into details on this, if you really are interested there is an entire industry dedicated to systems trading. Whether they work or not is another story.)
  • Forecasters need positive and negative feedback. When there is a question you have been thinking about, the worst thing you can do is make your forecast and then walk away when you are done. Forecasting requires constant iteration over the course of the questions lifecycle. The outcomes change constantly and the best thing the forecaster can do is constantly take in the feedback they are getting and apply it the best they can. Not listening to the feedback, even if it's negative can be detrimental to the overall outcome. Always take in the feedback and move forward from there. 
  • Prove yourself wrong. If there is a event of some sort that you are trying to prevent and the evidence seems to point overwhelmingly to a specific answer, try finding evidence that will prove that outcome wrong. Maybe it's just an opinion piece or from a new source that may not be entirely credible, but find it and analyze it anyway. As I stated in the last point, outcomes can change in the blink of an eye, even entirely predictable outcomes. But confirmation bias can be even more detrimental to a proper forecast analysis. Knowing all sides to a story is always a good bet. 

 It isn't just the financial industry that has been using forecasting successfully. Although Everyone from George Soros, Ray Dalio, to James Simmon's Renaissance Technologies are incredibly effective in short term (milliseconds for Ren-Tech) and longer term thinking (macro thinking application in the case of Dalio and Soros), other industries such as technology to sports are effectively using these techniques to model better outcomes for their businesses. The big five tech companies, Google, Apple, Facebook, Amazon, and Microsoft use and offer forecasting techniques in a myriad of ways. Google is by far the most innovative technology wise (type anything into the search input box and watch it "magically" try to predict what you are going to type), but all of these companies use forecasting in ways that weren't even possible 10 years ago. Amazon can predict what you are going to buy and Facebook can predict how you will react to a status update. That is just the basic level of forecasting these companies can accomplish. Then there is sports and more specifically the "Moneyball" techniques that were used in baseball with the Oakland A's and Boston Red Sox. (It helped that the Red Sox owner, John W. Henry owns a CTA fund and has successfully applied forecast and trend techniques to make money in the markets which is how he could afford to buy the Red Sox in the first place). I am convinced that other industries will start getting better at collecting and applying the data they get far more effectively.

Which leads me to write about how using the analytics techniques I mentioned above with more of the technological models (machine + deep learning) to get to effective forecasting. These two types of forecasting aren't mutually exclusive and should in fact be combined. In fact, that is probably one of the best ways that businesses (and human beings in particular) can really start to effectively use artificial intelligence; by using them together to try to forecast better results. Machine learning algorithms can model data faster and more effectively than a human being can but a human being can use quick judgement to understand when to use and not to use certain data. Weak A.I. is still the most prevalent in the world today (think Apples Siri, it's artificial intelligence, but it isn't very intelligent.) but we may start to see the emergence of stronger A.I. over the next 10 to 20 years (machines that can learn, apply, learn some more. Like how a child learns by doing.) As more and more data becomes available, there will more and more of a need to accurately forecast outcomes using that data. Data science and analysis plus applications of new technology in the field of A.I. will play a huge roll in pushing this forward.

From everything I have learned since I joined the tournament last August, I am extremely excited about what will be possible. For me, learning how to apply forecasting techniques has opened my eyes to my own capabilities and what I would like to do with them. I've always been fascinated by the emergence of A.I. and have wanted to somehow work in the field and am started to find ways to become more and more involved. I had already been pushing forward learning as much as I could about machine learning (yes, I took the Coursera class, but I still want to learn more applications) and since I have started to mine more open datasets, how these algorithms really work is becoming more clear to me. It is a fascinating and exciting field to be in and I can't wait to see what's to come.  

Truth

“When you’re young, you look at television and think, There’s a conspiracy. The networks have conspired to dumb us down. But when you get a little older, you realize that’s not true. The networks are in business to give people exactly what they want. That’s a far more depressing thought. Conspiracy is optimistic! You can shoot the bastards! We can have a revolution! But the networks are really in business to give people what they want. It’s the truth.”

- Steve Jobs

This quote is from 1996 but it still rings very true today, especially with algorithms controlling much of the content that we are seeing. It was important back then and doubly important now but we must learn to think critically for ourselves as opposed to just accepting what is being fed to us. Some of it may be true and some of it may be false, but the best we can do is make that decision for ourselves, not by a large company.