IBM And NVIDIA Power New Scale-Out Gear For AI

Accelerating deep learning (DL) training – on GPUs, TPUs, FPGAs or other accelerators – is in the early days scale-out architecture, like the server market was in the mid-2000s. DL training enables the advanced pattern recognition behind modern artificial intelligence (AI) based services. NVIDIA GPUs have been a major driver for DL development and commercialization, but IBM just made an important contribution to scale-out DL acceleration. Understanding what IBM did and how that work advances AI deployments takes some explanation.

Scale Matters

TIRIAS Research

Key Definitions

Inference scales-out. Trained DL models can be simplified for faster processing with good enough pattern recognition to create profitable services. Inference can scale-out as small individual tasks running on multiple inexpensive servers. There is a lot of industry investment aimed at lowering the cost of delivering inference, we’ll discuss that in the future.

The immediate challenge for creating deployable inference models is that, today, training scales-up. Training requires large data sets and high numeric precision; aggressive system designs are needed to meet real-world training times and accuracies. But cloud economics are driven by scale-out.

The challenge for cloud companies deploying DL-based AI services, such as Microsoft’s Cortana, Amazon’s Alexa and Google Home, is that DL training has not scaled well. Poor off-the-shelf scaling is mostly due to the immature state of DL acceleration, forcing service providers to invest (in aggregate) hundreds of millions of dollars in research and development (R&D), engineering and deployment of proprietary scale-out systems.

NVLink Scales-Up in Increments of Eight GPUs

GPU evolution has been a key part of DL success over recent years. General purpose processors were, and still are, too slow at processing DL math with large training data sets. NVIDIA invested early in leveraging GPUs for DL acceleration, in both new GPU architectures to further accelerate DL and in DL software development tools to enable easy access to GPU acceleration.

An important part of NVIDIA’s GPU acceleration strategy is NVLink. NVLink is a scale-up high-speed direct GPU-to-GPU interconnect architecture that directly connects two to eight GPU sockets. NVLink enables GPUs to train together with minimum processor intervention. Prior to NVLink, GPUs did not have the low-latency interconnect, data flow control sophistication, or unified memory space needed to scale-up by themselves. NVIDIA implements NVLink using its SXM2 socket instead of PCIe.

NVIDIA’s DGX-1, Microsoft’s Open Compute Project (OCP) Project Olympus HGX-1 GPU chassis and Facebook’s “Big Basin” server contribution to OCP are very similar designs that each house eight NVIDIA Tesla SXM2 GPUs. The DGX-1 design includes a dual-processor x86 server node in the chassis, while the HGX-1 and Big Basin designs must be paired with separate server chassis.

Microsoft’s HGX-1 can bridge four GPU chassis by using its PCIe switch chips to connect the four NVLink domains to one to four server nodes. While all three designs are significant feats of server architecture, the HGX-1’s 32-GPU design limit presents a practical upper limit for directly connected scale-up GPU systems.

TIRIAS Research

Microsoft HGX-1 motherboard with eight SXM2 sockets (four populated)

The list price for each DGX-1 is $129,000 using NVIDIA’s P100 SXM2 GPU and $149,000 using its V100 SXM2 GPU (including the built-in dual-processor x86 server node). While this price range is within reach of some high-performance computing (HPC) cluster bids, it is not a typical cloud or academic purchase.

Original Design Manufacturers (ODMs) like Quanta Cloud Technology (QCT) manufacture variants of OCP’s HGX-1 and Big Basin chassis, but do not publish pricing. NVIDIA P100 modules are priced from about $5,400 to $9,400 each. Because NVIDIA’s SXM2 GPUs account for most of the cost of both Big Basin and HGX-1, we believe that system pricing for both is in the range of $50,000 to $70,000 per chassis unit (not including matching x86 servers), in cloud-sized purchase quantities.

Facebook’s Big Basin Performance Claims

Facebook published a paper in June describing how it connected 32 Big Basin systems over its internal network to aggregate 256 GPUs and train a ResNet-50 image recognition model in under an hour with about 90% scaling efficiency and 72% accuracy.

While 90% scaling efficiency is an impressive achievement for state-of-the-art, there are several challenges with Facebook’s paper.

The eight-GPU Big Basin chassis is the largest possible scale-up NVIDIA NVLink instance. It is expensive, even if you could buy OCP gear as an enterprise buyer. Plus, Facebook’s paper does not mention which OCP server chassis design and processor model they used for their benchmarks. Which processor it used may be a moot point, because if you are not a cloud giant, it is very difficult to buy a Big Basin chassis or any of the other OCP servers that Facebook uses internally. Using different hardware, your mileage is guaranteed to vary.

Facebook also does not divulge the operating system or development tools used in the paper, because Facebook has its own internal cloud instances and development environments. No one else has access to them.

The net effect is that it is nearly impossible to replicate Facebook’s achievement if you are not Facebook.

TIRIAS Research

Facebook Big Basin Server

IBM Scales-Out with Four GPUs in a System

IBM recently published a paper as a follow-up to the Facebook paper. IBM’s paper describes how to train a Resnet-50 model in under an hour at 95% scaling efficiency and 75% accuracy, using the same data sets that Facebook used for training. IBM’s paper is notable in several ways:

  1. Not only did IBM beat Facebook on all the metrics, but 95% efficiency is very linear scaling.
  2. Anyone can buy the equipment and software to replicate IBM’s work. Equipment, operating systems and development environments are called out in the paper.
  3. IBM used smaller scale-out units than Facebook. Assuming Facebook used their standard dual-socket compute chassis, IBM has half the ratio of GPUs to CPUs – Facebook uses a 4:1 ratio and IBM uses a 2:1 ratio.

IBM sells its OpenPOWER “Minsky” deep learning reference design as the Power Systems S822LC for HPC. IBM’s PowerAI software platform with Distributed Deep Learning (DDL) libraries includes IBM-Caffe and “topology aware communication” libraries. PowerAI DDL is specific to OpenPOWER-based systems, so it will run on similar POWER8 Minsky-based designs and upcoming POWER9 “Zaius”-based systems (Zaius was designed by Google and Rackspace), such as those shown at various events by Wistron, E4, Inventec and Zoom.

PowerAI DDL enables creating large scale-out systems out of smaller, more affordable, GPU-based scale-up servers. It optimizes communications between GPU-based servers based on network topology, the capabilities of each network link, and the latencies for each phase of a DL model.

IBM used 64 Power System S822LC systems, each with four NVIDIA Tesla P100 SXM2-connected GPUs and two POWER8 processors, for a total of 256 GPUs – matching Facebook’s paper. Even with twice as many IBM GPU-accelerated chassis required to host the same number of GPUs as in Facebook’s system, IBM achieved a higher scaling efficiency than Facebook. That is no small feat.

TIRIAS Research

IBM Power System S822LC with two POWER8 processors (silver heat sinks) and four NVIDIA Tesla P100 SXM2 modules

Commercial availability of IBM’s S822LC for low volume buyers will be a key element enabling academic and enterprise researchers to buy a few systems and test IBM’s hardware and software scaling efficiencies. The base price for an IBM S822LC for Big Data (without GPUs) is $6,400, so the total price of a S822LC for High Performance Computing should be in the $30,000 to $50,000 ballpark (including the dual-processor POWER8 server node), depending on which P100 model is installed and other options.

Half the battle is knowing that something can be done. We believe IBM’s paper and product availability will spur a lot of DL development work by other hardware and software vendors.

— The author and members of the TIRIAS Research staff do not hold equity positions in any of the companies mentioned. TIRIAS Research tracks and consults for companies throughout the electronics ecosystem from semiconductors to systems and sensors to the cloud.

[“Source-forbes”]

United Continental Taps Apple and IBM for Mobile Tech Overhaul

PHOTO: EUROPEAN PRESSPHOTO AGENCY

United Continental Holdings Inc. plans a major overhaul of its mobile technology, which will include the development of new apps and the upgrade of the iPhones and iPads used by 50,000 flight and ground employees. The air carrier will work with Apple Inc. and International Business Machines Corp., which announced an alliance three years ago to develop business apps for Apple’s mobile devices.

The overhaul, which will include integration of the newly coded apps with older technology that manages customer and operations data, represents an effort by the airline to stop using one-off mobile apps and build a suite of apps, according to Jason Birnbaum, vice president of operations technology at United.

The company is looking for ways in which employees can make greater use of mobile technology. As part of the overhaul, the airline has defined at least 39 roles of employees involved in a typical flight, Mr. Birnbaum said in an interview. In one scenario, a gate agent waiting for a late-arriving passenger who is on the way from a connecting flight might query an app that the pilot has updated to determine if the door can be left open a few extra minutes, he said. Today, that information is often relayed by voice, with employees moving between plane, ramp and doorway. “We want to create a social community around a flight,” he said.

United expects this year to use some of the 12 iOS airline industry apps that IBM has built as part of its MobileFirst for iOS line. IBM and Apple announced the alliance in 2014, in an effort to sell more business apps, iPhones and iPads to corporate customers.

The airline also plans to write new apps with IBM that will be more tailored to United operations, he said. The intent is to give flight attendants, gate agents and other airline personnel access to data about customer histories and reservations, as well as real-time information about flight plans. In the event of a delay, for example, a flight attendant could use a mobile app to offer a specific passenger alternate connecting flights suited to his travel habits and rebook the ticket before touching down, Mr. Birnbaum said.

United last month experienced a computer glitch that affected hundreds of flights. It was the latest in a rash of computer failures disrupting U.S. air carriers.

Eventually, artificial intelligence via IBM’s Watson could help United further customize choices to individual flyers and enable natural language processing for agents or technicians . “In irregular operations, this is key,” said Dee Waddell, global managing director of travel and transportation industries at IBM. “They may not have time to type.”

United plans to issue iPhones for more customer service agents in airport terminals, especially because the wireless devices can help service unexpected crowds, Mr. Birnbaum said. Instead of agents being tied to PCs at podiums, they could fan out in gate areas during a surge in traffic during delays, he said.

Flight attendants at United typically use iPhone 6-Plus devices with apps for basic services such as ordering food. Now the airline wants to build a suite of in-flight apps that may, for example, guide attendants to suggest items to offer to specific customers during flights, Mr. Birnbaum said. An attendant who spills a drink on a flyer could use the app to add miles to the customer’s loyalty account as compensation, he said.

United expects to upgrade flight attendants to iPhone 7 models in another year, he estimated. The airline is testing the Apple Watch, but has no definite plans to deploy the wearable, he said. “We’re still trying to understand how they fit into the landscape for us. We feel like there’s an opportunity but haven’t found the perfect spot.”

[“source-ndtv”]

IBM Revenue Beats Estimates as Shift to Cloud Pays Off

IBM Revenue Beats Estimates as Shift to Cloud Pays Off

International Business Machines Corp’s quarterly revenue beat analysts’ expectations as the company’s shift to high-growth areas such as cloud-based services begins to yield results.

IBM also stood by its full-year forecast for adjusted earnings of at least $13.50 per share, dispelling any concerns about the impact from Britain’s vote to leave the European Union.

“Investors were a little bit nervous about the guidance, and they’ll find a little relief that the company maintained that, despite some headwinds associated with their high sales exposure to Europe,” said Edward Jones analyst Bill Kreher.

IBM receives nearly a third of its revenue from Europe, Middle East and Africa.

Chief Executive Ginni Rometty’s push towards cloud-based services, security software and data analytics seems to have paid off with a 12 percent rise in revenue from “strategic imperatives” in the second quarter.

Cloud revenue jumped 30 percent, compared with 34 percent in the preceding quarter.

“For us, it’s not about being the biggest cloud, that’s not our goal, our goal is to have the best hybrid capabilities,” Chief Financial Officer Martin Schroeter said in an interview.

Total revenue dropped 2.8 percent to $20.24 billion for the quarter ended June 30 from a year earlier, hurt by a fall in its traditional hardware businesses.

The company’s global business services revenue, which includes consulting, fell 2 percent, while its systems unit, which includes systems hardware, dropped 23.2 percent.

However, the company’s 17th straight quarterly revenue decline was not as steep as expected. The average analyst estimate was $20.02 billion, according to Thomson Reuters I/B/E/S.

Net income fell to $2.50 billion, or $2.61 per share, from $3.45 billion, or $3.50 per share.

Excluding items, IBM earned $2.95 per share, beating average analyst estimate of $2.89.

IBM’s shares, which had risen 16 percent this year through Monday, were up 0.7 percent in extended trading.

© Thomson Reuters 2016

Tags: Apps, Cloud, Cloud Computing, IBM, International Business Machines, Internet, Laptops, PCs, Watson
[“Source-Gadgets”]

IBM Teams With Apple on Artificial Intelligence Health Program

IBM Teams With Apple on Artificial Intelligence Health Program

IBM on Monday announced alliances with Apple and others to put artificial intelligence to work drawing potentially life-saving insights from the booming amount of health data generated on personal devices.

IBM is collaborating with Apple, Medtronic, and Johnson & Johnson to use its Watson artificial intelligence system to give users insights and advice from personal health information gathered from fitness trackers, smartphones, implants or other devices.

The initiative is trying to take advantage of medical records increasingly being digitized, allowing quick access for patients and healthcare providers if the information can be stored and shared effectively. IBM wants to create a platform for that sharing.

“All this data can be overwhelming for providers and patients alike, but it also presents an unprecedented opportunity to transform the ways in which we manage our health,” IBM senior vice president John Kelly said in a news release.

“We need better ways to tap into and analyze all of this information in real-time to benefit patients and to improve wellness globally.”

IBM expects more companies to join the health platform, which it envisions growing to a global scale.

In addition, the New York based company said it is acquiring a pair of healthcare technology companies and establishing an IBM health unit.

Watson is a cognitive computing system that bested human competition in a Jeopardy trivia television game show.

Under the partnership it will be able to handle data collected using health applications from Apple mobile devices, according to IBM.

“Now IBM’s secure cloud and analytics capabilities provide additional tools to help accelerate discoveries across a wide variety of health issues,” Apple senior vice president of operations Jeff Williams said in a release.

Download the Gadgets 360 app for Android and iOS to stay up to date with the latest tech news, product reviews, and exclusive deals on the popular mobiles.

Tags: Apple, Artificial Intelligence, IBM, Watson Supercomputer
[“source-ndtv”]