Nexedi, Europe's largest open-source software (OSS) publisher and creator of ERP5, installed a standard 19'' open rack with Tioga Pass servers in a China data center during the Covid-19 epidemic in March 2020 by cooperating remotely in real time with staff in two cities using Facepro Xpert glasses and systems.

First installation of Standard 19'' Open Rack in China Telecom Data Center

Nexedi bought two 19'' Rack ESA Kits with OCP Tioga Pass L10 servers for its Rapid Space site in Xinzhou, a city in Shanxi province in mainland China.

The 19' Rack ESA Kit and Tioga Pass servers are manufactured and supplied by Mitac from Taiwan. Rapid Space also uses Edgecore AS5812 Switches to allow all OCP servers to be connected via 10 Gbps LAN with 1Gbps Internet transit. Mitac shipped all equipment to the China Telecom data center in Xinzhou at the end of 2019 and Nexedi planned to send their Chinese staff in the beginning of 2020 to assemble and install all machines.

Challenge: Covid-19 + Complete Lockdown

In January 2020, Covid-19 started spreading in Wuhan and very soon the virus was detected all around China. By the end of January, Wuhan was completely locked down and other cities were shutting down one-by-one. People were asked to stay in their homes, all unnecessary activities were prohibited, and travel between cities, provinces or districts was strictly forbidden. All business activity in China stopped. All flights from foreign countries were stopped with no date set for restarting schedules.

One Chinese developer of Nexedi was stuck in his hometown of Anhui. All other Nexedians capable of assembling or installing the servers were in foreign countries. Only staff from our Rapid Space local partner had access to the site but without prior experience in assembling or installing the servers.

The schedule for the installation was therefore completely stopped.

Remote Installation: real-time cooperation between France and China

In March Edge-Core introduced Softfoundry's FacePro Xpert System to Nexedi, a tool designed for technicians and engineers to assist in remote on-site service, equipment inspection, maintenance and complex manufacturing assembly. A system is composed of one or more experts working on a PC, tablet or smart phone and a field engineer with smart glasses. Participants can connect to the system using any type of network in order to communicate and visualize the machine and situation in real time. Without knowing when the lock down would be lifted in China, and the Covid-19 situation in France and other countries outside China deteriorating, Nexedi decided to adopt this FacePro Xpert system to handle the assembly and installation.

The staff onsite brought all the servers and pieces to the data center and wore the Xpert smart glasses to join a visual meeting with Nexedi.

In a first step, rails to the data center cabinet were fixed with screws before installing a BUS bar. Once panels (both left and right sides) and the BUS bar were installed, the rack (shelf) was inserted and mounted on the side panels.

Next a power supply shelf was installed and fixed on the cabinet to ensure that the power supply module would connect to the BUS bar before inserting the power supply unit itself. After assembling the switch with its pendants and rails by following the manual from Edgecore, staff fixed the switch to the cabinet, as pointed out by the red arrow in the freezed view.

Afterwards servers were plugged into each rack one by one, powered on and connected to the switch to complete the server installation.

Nexedi's engineer in Anhui joined the meeting with his laptop.

Nexedi's engineers in France joined the meeting with their laptops.

During the meeting, Nexedi engineers (in France, Poland and in China) were able to get an on-site first person, real-time view and at the same time communicate with the staff transmitting video and audio to give instructions. To assist the on-site staff, the FacePro Xpert System was used to freeze certain views to mark or highlight specific areas for the on-site staff to identify. The field staff had their hands free to assemble machines while reading instruction on the glasses and listening to direct advice from Nexedi experts. They could also voice control the view on the glasses to better communicate and present their view and questions to Nexedi.

With this real-time cooperation and sharing of first-person perspective, we are able to complete the assembly of all servers and installation of required software.

In order to configure the AS5812 switch to make computers communicate, we installed AOS (ACCTON Operating System for switches). AOS will be soon open sourced by Edgecore.

For every machine, we also pre-installed remotely a Debian 10 installation image, from which users can easily install Debian 10 server operating system. Nevertheless, they are also free to install any other server operating systems for their VM according to their needs.

Nexedi used Raspberry Pi with IPv6 to remote operate and control the installation. First, we installed Re6st (to provide IPv6 access in China thanks to Grandenet) on a Raspberry Pi. Then the on-site staff turned on a mobile hotspot and made the Raspberry Pi connect to the internet by using the hotspot. After that the Raspberry Pi was connected to the debug port of the server with a network cable. With this method, Nexedi engineer could log in to the Raspberry Pi remotely through ssh.

Success factors: Full tutorial + Facepro Xpert System (smart glasses)

We are very happy with the result of this remote corporation assembly activity. There are four main factors responsible for this success:

· Detailed step-by-step tutorials ad procedures are available to on-site staff to consult in advance, or even during the assembly, because we can also share documents while using the FacePro Xpert System.
· The FacePro Xpert System itself with smart glasses solve the problem of sharing the first-person view while experts are thousand kilometers away from the site.
· Use the Raspberry Pi as a bridge to access the server to diagnose or debug.
· Re6st allows to give each server a unique IPv6 address, so that all servers are able to expose services on IPv6.

We strongly think this is a solution that could be widely adopted in many industries that need remote corporation on some engineering activities.