This Cisco Validated Design describes how the Cisco Unified Computing System™ can be used in conjunction with EMC® VNX™ storage systems to. Cisco Collaboration. Mustafa Jarrar. Systems Engineer – Saudi Arabia. Unified Communications – Real Estate. “Why can't I work from home?” “Let me use. Why won't you support my iPhone?” Partners. “Give me equal.

Access to case studies expires six months after purchase date. Publication Date: June 05, 2009 In November 2007, a global, cross-functional team at Cisco Systems, Inc. Was seeking management approval to start manufacturing a new router, code-named Viking. The team faced a host of challenges in launching the low-cost but powerful router for telecommunications service providers. After overhauling the project to sharply increase the router's planned speed and capacity, the company had just one year to launch the product, an unusually fast schedule.

In addition, Cisco wanted to debut China as a low-cost manufacturing base for the high-end product. It planned to use contract manufacturer Foxconn Technology Group to produce the machine, even though Foxconn had never made such a complex product for Cisco. Could Foxconn handle the technical complexity? Could Cisco work closely with Foxconn to mitigate the project risks? Could Cisco's methodology for new product introduction rise to the necessary level of sophistication?

The case highlights the challenges and complexities of developing and manufacturing a sophisticated technology product for a worldwide market. Students will consider what it takes to achieve success in new product introduction, or NPI. The case also offers an opportunity to evaluate supply chain issues in a company that outsources manufacturing globally. When you place your first order on HBR.org and enter your credit card information and shipping address, 'Speed-Pay' ordering is enabled. 'Speed-Pay' is a service that saves the credit card details from your most recent purchase and allows you to re-use that card for future purchases.

If you click the Speed-Pay button on any product detail page, your order will be charged to the most recent credit card information attached to your account and shipped (if applicable) to the last address we have on file for you. Ebook: A digital book provided in three formats (PDF, ePub, and Mobi) for the price of one. Accessible within “My Library” upon purchase. Hardcopy, paperback, softbound, magazine: Physical copy shipped from our warehouse to your requested shipping location. PDF: PDF digital file. Accessible within “My Library” upon purchase.

Bundle: A themed collection containing two or more items at a special savings. Note: Some of our products are available in other languages besides English, for example a “Spanish PDF” format means you will receive a PDF in the Spanish language. A security code is added protection against credit card fraud. It is a 3 or 4 digit number appearing on the front or back of your credit card. See examples below. Visa and Mastercard The security code has 3 digits and appears on the BACK of the card in the signature panel.

American Express The Card Identification Number (CID/4DBC) is a four-digit, non-embossed (flat) number that is printed on every American Express Card. The CID/4DBC is ALWAYS located ABOVE the embossed (raised) account number on the face of the card. In some instances, the CID/4DBC is located on the left side of the card, but is always above the account number. Some cards have a four-digit number embossed below the account number, but this is not the CID/4DBC.

Cisco UCS Storage Server with Scality Ring Design and Deployment of Scality Object Storage on Cisco UCS S3260 Storage Server NOTE: Works with document’s Advanced Properties “First Published” property. Click File Properties Advanced Properties Custom. Last Updated: April 10, 2017 NOTE: Works with document’s Advanced Properties “Last Updated” property. Click File Properties Advanced Properties Custom. About the Cisco Validated Design (CVD) Program The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments.

For more information visit. ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, 'DESIGNS') IN THIS MANUAL ARE PRESENTED 'AS IS,' WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE.

USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO. Modern data centers increasingly rely on a variety of architectures for storage. Whereas in the past organizations focused on block and file storage only, today organizations are focusing on object storage, for several reasons: Object storage offers unlimited scalability and simple management Because of the low cost per gigabyte, object storage is well suited for large-capacity needs, and therefore for use cases such as archive, backup, and cloud operations Object storage allows the use of custom metadata for objects Enterprise storage systems are designed to address business-critical requirements in the data center. But these solutions may not be optimal for use cases such as backup and archive workloads and other unstructured data, for which data latency is not especially important.

Scality object Storage is a massively scalable, software-defined storage system that gives you unified storage for your cloud environment. It is an object storage architecture that can easily achieve enterprise-class reliability, scale-out capacity, and lower costs with an industry-standard server solution. The Cisco UCS S3260 Storage Server, originally designed for the data center, together with Scality RING is optimized for object storage solutions, making it an excellent fit for unstructured data workloads such as backup, archive, and cloud data. The S3260 delivers a complete infrastructure with exceptional scalability for computing and storage resources together with 40 Gigabit Ethernet networking.

The S3260 is the platform of choice for object storage solutions because it provides more than comparable platforms: Proven server architecture that allows you to upgrade individual components without the need for migration High-bandwidth networking that meets the needs of large-scale object storage solutions like Scality RING Storage Unified, embedded management for easy-to-scale infrastructure Cisco and Scality are collaborating to offer customers a scalable object storage solution for unstructured data that is integrated with Scality RING Storage. With the power of the Cisco UCS management framework, the solution is cost effective to deploy and manage and will enable the next-generation cloud deployments that drive business agility, lower operational costs and avoid vendor lock-in. Traditional storage systems are limited in their ability to easily and cost-effectively scale to support massive amounts of unstructured data. With about 80 percent of data being unstructured, new approaches using x86 servers are proving to be more cost effective, providing storage that can be expanded as easily as your data grows. Object storage is the newest approach for handling massive amounts of data. Scality is an industry leader in enterprise-class, petabyte-scale storage.

Scality introduced a revolutionary software-defined storage platform that could easily manage exponential data growth, ensure high availability, deliver high performance and reduce operational cost. Scality’s scale-out storage solution, the Scality RING, is based on patented object storage technology and operates seamlessly on any commodity server hardware. It delivers outstanding scalability and data persistence, while its end-to-end parallel architecture provides unsurpassed performance. Scality’s storage infrastructure integrates seamlessly with applications through standard storage protocols such as NFS, SMB and S3. Scale-out object storage uses x86 architecture storage-optimized servers to increase performance while reducing costs.

The Cisco UCS S3260 Storage Server is well suited for object-storage solutions. It provides a platform that is cost effective to deploy and manage using the power of the Cisco Unified Computing System (Cisco UCS) management: capabilities that traditional unmanaged and agent-based management systems can’t offer. You can design S3260 solutions for a computing-intensive, capacity-intensive, or throughput-intensive workload.

Both solutions together, Scality object Storage and Cisco UCS S3260 Storage Server, deliver a simple, fast and scalable architecture for enterprise scale-out storage. The current Cisco Validated Design (CVD) is a simple and linearly scalable architecture that provides object storage solution on Scality RING and Cisco UCS S3260 Storage Server. The Cisco Unified Computing System (Cisco UCS) is a state-of-the-art data center platform that unites computing, network, storage access, and virtualization into a single cohesive system. The main components of Cisco Unified Computing System are: Computing - The system is based on an entirely new class of computing system that incorporates rack-mount and blade servers based on Intel Xeon Processor E5 and E7.

The Cisco UCS servers offer the patented Cisco Extended Memory Technology to support applications with large datasets and allow more virtual machines (VM) per server. Network - The system is integrated onto a low-latency, lossless, 40-Gbps unified network fabric. This network foundation consolidates LANs, SANs, and high-performance computing networks which are separate networks today. The unified fabric lowers costs by reducing the number of network adapters, switches, and cables, and by decreasing the power and cooling requirements. Virtualization - The system unleashes the full potential of virtualization by enhancing the scalability, performance, and operational control of virtual environments.

Cisco security, policy enforcement, and diagnostic features are now extended into virtualized environments to better support changing business and IT requirements. Storage access - The system provides consolidated access to both SAN storage and Network Attached Storage (NAS) over the unified fabric. By unifying the storage access the Cisco Unified Computing System can access storage over Ethernet (NFS or iSCSI), Fibre Channel, and Fibre Channel over Ethernet (FCoE). This provides customers with choice for storage access and investment protection. In addition, the server administrators can pre-assign storage-access policies for system connectivity to storage resources, simplifying storage connectivity, and management for increased productivity.

The Cisco Unified Computing System is designed to deliver: A reduced Total Cost of Ownership (TCO) and increased business agility. Increased IT staff productivity through just-in-time provisioning and mobility support. A cohesive, integrated system which unifies the technology in the data center. Industry standards supported by a partner ecosystem of industry leaders.

S3260 Storage Server The Cisco UCS® S3260 Storage Server (Figure 1) is a modular, high-density, high-availability dual node rack server well suited for service providers, enterprises, and industry-specific environments. It addresses the need for dense cost effective storage for the ever-growing data needs.

Designed for a new class of cloud-scale applications, it is simple to deploy and excellent for big data applications, software-defined storage environments and other unstructured data repositories, media streaming, and content distribution. Extending the capability of the Cisco UCS S-Series portfolio, the Cisco UCS S3260 helps you achieve the highest levels of data availability. With dual-node capability that is based on the Intel® Xeon® processor E5-2600 v4 series, it features up to 600 TB of local storage in a compact 4-rack-unit (4RU) form factor.

All hard-disk drives can be asymmetrically split between the dual-nodes and are individually hot-swappable. The drives can be built-in in an enterprise-class Redundant Array of Independent Disks (RAID) redundancy or be in a pass-through mode. This high-density rack server comfortably fits in a standard 32-inch depth rack, such as the Cisco® R42610 Rack. The Cisco UCS S3260 is deployed as a standalone server in both bare-metal or virtualized environments. Its modular architecture reduces total cost of ownership (TCO) by allowing you to upgrade individual components over time and as use cases evolve, without having to replace the entire system. The Cisco UCS S3260 uses a modular server architecture that, using Cisco’s blade technology expertise, allows you to upgrade the computing or network nodes in the system without the need to migrate data migration from one system to another. The reference architecture use case provides a comprehensive, end-to-end example of deploying Scality object storage on Cisco UCS S3260 (Figure 8).

The first section in this Cisco Validated Design covers setting up the Cisco UCS hardware; the Cisco UCS 6332 Fabric Interconnects (Cisco UCS Manager), Cisco UCS S3260 Storage servers, Cisco UCS C220 M4 Rack Servers, and the peripherals like Cisco Nexus 9332 switches. The second section explains the step-by-step installation instructions to install Scality RING. The final section includes the functional and High Availability tests on the test bed, performance, and the best practices evolved while validating the solution. Figure 8 Cisco UCS SDS Architecture The current solution based on Cisco UCS and Scality object Storage is divided into multiple sections and covers three main aspects. This CVD describes the architecture, design and deployment of a Scality object Storage solution on six Cisco UCS S3260 Storage Server, each with two Cisco UCS C3X60 M4 nodes configured as Storage servers and 3 Cisco UCS C220 M4S Rack servers as three Connector nodes and one Supervisor node. The whole solution is connected to the pair of Cisco UCS 6332 Fabric Interconnects and to pair of upstream network switch Cisco Nexus 9332PQ.

The detailed configuration is as follows: Two Cisco Nexus 9332PQ Switches Two Cisco UCS 6332 Fabric Interconnects Six Cisco UCS S3260 Storage Servers with two UCS C3X60 M4 server nodes each Three Cisco UCS C220 M4S Rack Servers. Layer Component Version or Release Storage (Chassis) UCS S3260 Chassis Management Controller 2.0(13e) Shared Adapter 4.1(2d) Compute (Server Nodes) UCS C3X60 M4 BIOS C3x60M4.2.0.13c CIMC Controller 2.0(13f) Compute (Rack Server) C220 M4S BIOS C220M4.2.0.13d CIMC Controller 2.0(13f) Network 6332 Fabric Interconnect UCS Manager 3.1(2b) Kernel 5.0(3)N2(3.12b) System 5.0(3)N2(3.12b) Network Nexus 9332PQ BIOS 07.59 NXOS 7.0(3)I5(1) Software Red Hat Enterprise Linux Server 7.3 (x86_64) Scality RING 6.3 This section contains the hardware components (Table 2) used in the test bed. This section provides the details for configuring a fully redundant, highly available Cisco UCS 6332 fabric configuration.

Initial setup of the Fabric Interconnect A and B Connect to Cisco UCS Manager using virtual IP address of using the web browser Launch Cisco UCS Manager Enable server and uplink ports Start discovery process Create pools and policies for service profile template Create chassis and storage profiles Create Service Profile templates and appropriate Service Profiles Associate Service Profiles to servers To set up the Cisco UCS 6332 Fabric Interconnects A and B, complete the following steps: 1. Connect to the console port on the first Cisco UCS 6332 Fabric Interconnect. At the prompt to enter the configuration method, enter console to continue. If asked to either perform a new setup or restore from backup, enter setup to continue. Enter y to continue to set up a new Fabric Interconnect. Enter n to enforce strong passwords.

Enter the password for the admin user. Enter the same password again to confirm the password for the admin user.

When asked if this fabric interconnect is part of a cluster, answer y to continue. Enter A for the switch fabric. Enter the cluster name UCS- FI-6332 for the system name.

Enter the Mgmt0 IPv4 address. Enter the Mgmt0 IPv4 netmask. Enter the IPv4 address of the default gateway.

Enter the cluster IPv4 address. To configure DNS, answer y. Enter the DNS IPv4 address. Answer y to set up the default domain name. Enter the default domain name.

Review the settings that were printed to the console, and if they are correct, answer yes to save the configuration. Wait for the login prompt to make sure the configuration has been saved.

---- Basic System Configuration Dialog ---- This setup utility will guide you through the basic configuration of the system. Only minimal configuration including IP connectivity to the Fabric interconnect and its clustering mode is performed through these steps. Type Ctrl-C at any time to abort configuration and reboot system. To back track or make modifications to already entered values, complete input till end of section and answer no when prompted to apply configuration. Enter the configuration method. Console Enter the setup mode; setup newly or restore from backup. Setup You have chosen to setup a new Fabric interconnect.

(y/n): y Enforce strong password? (y/n) [y]: n Enter the password for 'admin': Confirm the password for 'admin': Is this Fabric interconnect part of a cluster(select 'no' for standalone)? (yes/no) [n]: yes Enter the switch fabric (A/B): A Enter the system name: UCS-FI-6332 Physical Switch Mgmt0 IP address: 192.168.10.101 Physical Switch Mgmt0 IPv4 netmask: 255.255.255.0 IPv4 address of the default gateway: 192.168.10.1 Cluster IPv4 address: 192.168.10.100 Configure the DNS Server IP address? (yes/no) [n]: no Configure the default domain name? (yes/no) [n]: no Join centralized management environment (UCS Central)?

(yes/no) [n]: no Following configurations will be applied: Switch Fabric=A System Name= UCS-FI-6332 Enforced Strong Password=no Physical Switch Mgmt0 IP Address=192.168.10.101 Physical Switch Mgmt0 IP Netmask=255.255.255.0 Default Gateway=192.168.10.1 Ipv6 value=0 Cluster Enabled=yes Cluster IP Address=192.168.10.100 NOTE: Cluster IP will be configured only after both Fabric Interconnects are initialized. UCSM will be functional only after peer FI is configured in clustering mode. Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes Applying configuration. Configuration file - Ok Cisco UCS 6300 Series Fabric Interconnect UCS-FI-6332-A login: 1. Connect to the console port on the second Cisco UCS 6332 Fabric Interconnect. When prompted to enter the configuration method, enter console to continue.

The installer detects the presence of the partner Fabric Interconnect and adds this fabric interconnect to the cluster. Enter y to continue the installation.

Enter the admin password that was configured for the first Fabric Interconnect. Enter the Mgmt0 IPv4 address. Answer yes to save the configuration. Wait for the login prompt to confirm that the configuration has been saved. ---- Basic System Configuration Dialog ---- This setup utility will guide you through the basic configuration of the system. Only minimal configuration including IP connectivity to the Fabric interconnect and its clustering mode is performed through these steps. Type Ctrl-C at any time to abort configuration and reboot system.

To back track or make modifications to already entered values, complete input till end of section and answer no when prompted to apply configuration. Enter the configuration method. Console Installer has detected the presence of a peer Fabric interconnect. This Fabric interconnect will be added to the cluster. Continue (y/n)? Y Enter the admin password of the peer Fabric interconnect: Connecting to peer Fabric interconnect.

Done Retrieving config from peer Fabric interconnect. Done Peer Fabric interconnect Mgmt0 IPv4 Address: 192.168.10.101 Peer Fabric interconnect Mgmt0 IPv4 Netmask: 255.255.255.0 Cluster IPv4 address: 192.168.10.100 Peer FI is IPv4 Cluster enabled. Please Provide Local Fabric Interconnect Mgmt0 IPv4 Address Physical Switch Mgmt0 IP address: 192.168.10.102 Apply and save the configuration (select 'no' if you want to re-enter)? (yes/no): yes Applying configuration. Configuration file - Ok Cisco UCS 6300 Series Fabric Interconnect UCS-FI-6332-B login: To login to Cisco UCS Manager, complete the following steps: 1. Open a Web browser and navigate to the Cisco UCS 6332 Fabric Interconnect cluster address. Click the Launch link to download the Cisco UCS Manager software.

If prompted to accept security certificates, accept as necessary. Click Launch UCS Manager HTML. When prompted, enter admin for the username and enter the administrative password.

Click Login to log in to the Cisco UCS Manager. This section describes how to configure the NTP server for the Cisco UCS environment. Select Admin tab on the left site. Select Time Zone Management. Select Time Zone. Under Properties select your time zone.

Select Add NTP Server. Enter the IP address of the NTP server. Figure 12 Adding a NTP server - Summary This section describes how to configure the global policies. Select the Equipment tab on the left site of the window. Select Policies on the right site. Select Global Policies. Under Chassis/FEX Discovery Policy select Platform Max under Action.

Select 40G under Backplane Speed Preference. Under Rack Server Discovery Policy select Immediate under Action. Under Rack Management Connection Policy select Auto Acknowledged under Action.

Under Power Policy select Redundancy N+1. Under Global Power Allocation Policy select Policy Driven. Select Save Changes. Figure 13 Configuration of Global Policies To enable server ports, complete the following steps: 1. Select the Equipment tab on the left site. Select Equipment >Fabric Interconnects >Fabric Interconnect A (subordinate) >Fixed Module.

Click Ethernet Ports section. Select Ports 1-12, right-click and then select Configure as Server Port and click Yes and then OK. Select Ports 17-20 for C220 M4S server, right-click and then select “ Configure as Server Port” and click Yesand then OK.

Repeat the same steps for Fabric Interconnect B. Figure 14 Configuration of Server Ports To enable uplink ports, complete the following steps: 1. Select the Equipment tab on the left site. Select Equipment >Fabric Interconnects >Fabric Interconnect A (subordinate) >Fixed Module. Click Ethernet Ports section. Select Ports 25-26, right-click and then select Configure as Uplink Port.

Click Yes and then OK. Repeat the same steps for Fabric Interconnect B. To label each server (provides better identification), complete the following steps: 1. Select the Equipment tab on the left site. Select Chassis >Chassis 1 >Server 1.

In the Properties section on the right go to User Label and add Storage-Node1 to the field. Repeat the previous steps for Server 2 of Chassis 1 and for all other servers of Chassis 2 – 6 according to Table 2.

Go to Servers >Rack-Mounts >Servers >and repeat the step for all servers according to Table 4. VLAN Name Function 10 Storage-Management Storage Management traffic for Supervisor, Connector & Storage Nodes 20 Storage-Cluster Storage Cluster traffic for Supervisor, Connector & Storage Nodes 30 Client-Network (optional) Client traffic for Connector & Storage Nodes 79 External-Network External Public Network for all UCS Servers To configure VLANs in the Cisco UCS Manager GUI, complete the following steps: 1. Select LAN in the left pane in the Cisco UCS Manager GUI. Select LAN >LAN Cloud >VLANs and right-click Create VLANs. Enter Storage-Management for the VLAN Name. Keep Multicast Policy Name as. Select Common/Global for Public.

Enter 10 in the VLAN IDs field. Click OK and then Finish. Figure 21 Create a VLAN 8.

Repeat the steps for the rest of the VLANs Storage-Cluster, Client Network, and External-Network. To enable Network Control Policies, complete the following steps: 1. Select the LAN tab in the left pane of the Cisco UCS Manager GUI. Go to LAN >Policies >root >Network Control Policies and right-click Create Network-Control Policy. Type in Enable-CDP in the Name field. (Optional) Enter a description in the Description field. Click Enabled under CDP.

Click All Hosts VLANs under MAC Register Mode. Leave everything else untouched and click OK.

Figure 22 Create a Network Control Policy To create a Quality of Service System Class, complete the following steps: 1. Select the LAN tab in the left pane of the Cisco UCS Manager GUI.

Go to LAN >LAN Cloud >QoS System Class. Enable Priority Platinum & Gold and set Weight 10 & 9 respectively and MTU to 9216 and Best Effort MTU as 9216. Set Fibre Channel Weight to None. Click Save Changes and then OK. Figure 23 QoS System Class Based on the previous QoS System Class, setup a QoS Policy with the following configuration: 1. Select the LAN tab in the left pane of the Cisco UCS Manager GUI. Go to LAN >Policies >root >QoS Policies and right-click Create QoS Policy.

Type in Storage-Mgmt in the Name field. Set Priority as platinum and leave everything else unchanged. Click OK and then OK. Figure 24 QoS Policy Setup 6. Repeat the steps to create Qos Policy for Storage-Cluster and Set Priority as Gold.

Based on the previous section, creating VLANs, the next step is to create the appropriate vNIC templates. For Scality Storage we need to create four different vNICs, depending on the role of the server. Table 6 provides an overview of the configuration. Name vNIC Name Fabric Interconnect Failover VLAN MTU Size MAC Pool Network Control Policy Storage-Mgmt Storage-Mgmt A Yes Storage-Mgmt – 10 9000 UCS-MAC-Pools Enable-CDP Storage-Cluster Storage-Cluster B Yes Storage-Cluster - 20 9000 UCS-MAC-Pools Enable-CDP Client-Network Client-Network A Yes Client-Network – 30 1500 UCS-MAC-Pools Enable-CDP External-Mgmt External-Mgmt A Yes External-Mgmt -79 1500 UCS-MAC-Pools Enable-CDP To create the appropriate vNICs, complete the following steps: 1.

Select the LAN tab in the left pane of the Cisco UCS Manager GUI. Go to LAN >Policies >root >vNIC Templates and right-click Create vNIC Template. Type in Storage-Mgmt in the Name field. (Optional) Enter a description in the Description field.

Click Fabric A as Fabric ID and enable failover. Select default as VLANs and click Native VLAN. Select UCS-MAC-Poolsas MAC Pool. Select Storage-Mgmt as QoS Policy. Select Enable-CDP as Network Control Policy. Click OK and then OK.

Figure 25 Setup the vNIC Template for Storage-Mgmt vNIC 11. Repeat the steps for the vNICs Storage-Cluster, Client-NIC and External-Mgmt. Make sure you select the correct Fabric ID, VLAN and MTU size according to Table 6. By default, Cisco UCS provides a set of Ethernet adapter policies. These policies include the recommended settings for each supported server operating system. Operating systems are sensitive to the settings in these policies. Note: Cisco UCS best practice is to enable Jumbo Frames MTU 9000 for any Storage facing Networks (Storage-Mgmt & Storage-Cluster). Enabling jumbo frames on specific interfaces, guarantees 39Gb/s bandwidth on the Cisco UCS fabric.

For Jumbo Frames MTU9000, you can use default Ethernet Adapter Policy predefined as Linux. If the customer deployment scenarios only supports only MTU1500, you can still modify the Ethernet Adapter policy resources Tx & Rx queues to guarantee 39Gb/s bandwidth. To create a specific adapter policy for Red Hat Enterprise Linux, complete the following steps: 1. Select the Server tab in the left pane of the Cisco UCS Manager GUI.

Go to Servers >Policies >root >Adapter Policies and right-click Create Ethernet Adapter Policy. Type in RHEL in the Name field. (Optional) Enter a description in the Description field. Under Resources type in the following values: a. Transmit Queues: 8 b. Ring Size: 4096 c.

Receive Queues: 8 d. Ring Size: 4096 e. Completion Queues: 16 f. Interrupts: 32 6. Under Options enable Receive Side Scaling (RSS). Click OK and then OK.

Figure 26 Adapter Policy for RHEL To create a Boot Policy, complete the following steps: 1. Select the Servers tab in the left pane. Go to Servers >Policies >root >Boot Policies and right-click Create Boot Policy. Type in a Local-OS-Boot in the Name field. (Optional) Enter a description in the Description field.

Figure 27 Create Boot Policy 5. Click Local Devices >Add Local CD/DVD and click OK. Click Local Devices >Add Local LUN and Set Type as “Any” and click OK. To create a LAN Connectivity Policy, complete the following steps: 1.

Select the LAN tab in the left pane. Go to Servers >Policies >root >LAN Connectivity Policies and right-click Create LAN Connectivity Policy for Storage Servers. Type in Storage-Node in the Name field. (Optional) Enter a description in the Description field. Figure 28 LAN Connectivity Policy 6. Type in Storage-Mgmt in the name field.

Click “Use vNIC Template.” 8. Select vNIC template for “Storage-Mgmt” from drop-down list. If you are using Jumbo Frame MTU 9000, Select default Adapter Policy as Linux from the drop-down list. Note: If you are using MTU 1500, Select Adapter Policy as RHEL created before from the drop-down list.

Figure 29 LAN Connectivity Policy 10. Repeat the vNIC creation steps for Storage-Cluster, Client-Network, and External-Network. To setup a Maintenance Policy, complete the following steps: 1. Select the Servers tab in the left pane. Go to Servers >Policies >root >Maintenance Policies and right-click Create Maintenance Policy.

Type in a Server-Maint in the Name field. (Optional) Enter a description in the Description field. Click User Ack under Reboot Policy.

Click OK and then OK. Create Maintenance Policy. To create a Power Control Policy, complete the following steps: 8. Select the Servers tab in the left pane. Go to Servers >Policies >root >Power Control Policies and right-click Create Power Control Policy.

Type in No-Power-Cap in the Name field. (Optional) Enter a description in the Description field. Click No Cap and click OK. Create Power Control Policy. The Chassis Profile is required to assign specific disks to a particular server node in a Cisco UCS S3260 Storage Server as well as upgrading to a specific chassis firmware package.

To create a Chassis Firmware Package, complete the following steps: 1. Select the Chassis tab in the left pane of the Cisco UCS Manager GUI. Go to Chassis >Policies >root >Chassis Firmware Package and right-click Create Chassis Firmware Package. Type in UCS-S3260-FW in the Name field. (Optional) Enter a description in the Description field.

Select 3.1.(2b)C form the drop-down menu of Chassis Package. Select OK and then OK. Create Chassis Firmware Package. To create a Chassis Maintenance Policy, complete the following steps: 1.

Select the Chassis tab in the left pane of the Cisco UCS Manager GUI. Go to Chassis >Policies >root >Chassis Maintenance Policies and right-click Create Chassis Maintenance Policy. Type in UCS-S3260-Main in the Name field. (Optional) Enter a description in the Description field. Click OK and then OK.

Create Chassis Maintenance Policy. To create a Disk Zoning Policy, complete the following steps: 7. Select the Chassis tab in the left pane of the Cisco UCS Manager GUI. Go to Chassis >Policies >root >Disk Zoning Policies and right-click Create Disk Zoning Policy.

Type in UCS-S3260-Zoning in the Name field. (Optional) Enter a description in the Description field. Create Disk Zoning Policy. Select Dedicated under Ownership. Select Server 1 and Select Controller 1. Add Slot Range 1-28 for the top node of the Cisco UCS S3260 Storage Server and click OK. Add Slots to Top Node of Cisco UCS S3260.

Select Dedicated under Ownership. Select Server 2 and Select Controller 1. Add Slot Range 29-56 for the bottom node of the Cisco UCS S3260 Storage Server and click OK. Add Slots to Bottom Node of Cisco UCS S3260. To create a Chassis Profile Template, complete the following steps: 22.

Select the Chassis tab in the left pane of the Cisco UCS Manager GUI. Go to Chassis >Chassis Profile Templates and right-click Create Chassis Profile Template. Type in S3260-Chassis in the Name field. Under Type, select Updating Template. (Optional) Enter a description in the Description field. Create Chassis Profile Template. Under the radio button Chassis Maintenance Policy, select your previously created Chassis Maintenance Policy.

Chassis Profile Template – Chassis Maintenance Policy. Select the + button and select under Chassis Firmware Package your previously created Chassis Firmware Package Policy. Chassis Profile Template – Chassis Firmware Package. Under Disk Zoning Policy select your previously created Disk Zoning Policy.

Chassis Profile Template – Disk Zoning Policy 37. Click Finish and then click OK. To create the Chassis Profiles from the previous created Chassis Profile Template, complete the following steps: 1. Select the Chassis tab in the left pane of the Cisco UCS Manager GUI. Go to Chassis >Chassis Profiles and right-click Create Chassis Profiles from Template. Type in S3260-Chassis in the Name field.

Leave the Name Suffix Starting Number untouched. Enter 6 for the Number of Instances for all connected Cisco UCS S3260 Storage Server.

Choose your previously created Chassis Profile Template. Click OK and then click OK. Create Chassis Profiles from Template. To associate all previous created Chassis Profile, complete the following steps: 1. Select the Chassis tab in the left pane of the Cisco UCS Manager GUI.

Go to Chassis >Chassis Profiles and select S3260-Chassis. Right-click Change Chassis Profile Association. Under Chassis Assignment, choose Select existing Chassis. Under Available Chassis, select ID 1. Click OK and then click OK again. Repeat the steps for the other four Chassis Profiles by selecting the IDs 2 – 6.

Associate Chassis Profile. To prepare all disks from the Rack-Mount Servers for storage profiles, the disks have to be converted from JBOD to Unconfigured-Good.

To convert the disks, complete the following steps: 1. Select the Equipment tab in the left pane of the Cisco UCS Manager GUI. Go to Equipment >Rack-Mounts >Servers >Server 1 >Disks. Select both disks and right-click Set JBOD to Unconfigured-Good. Repeat the steps for Server 2-4. Set Disks for C220 M4 Servers to Unconfigured-Good. To create the Storage Profile for Boot LUNs for the top node of the Cisco UCS S3260 Storage Server, complete the following steps: 1.

Select Storage in the left pane of the Cisco UCS Manager GUI. Go to Storage >Storage Profiles and right-click Create Storage Profile. Type in S3260-OS-Node1 in the Name field.

(Optional) Enter a description in the Description field. Type in OS-BootLUN in the Name field. Configure as follow: a. Create Local LUN b. Size (GB) = 1 c.

Fractional Size (MB) = 0 d. Auto Deploy e.

Select Expand To Available f. Click Create Disk Group Policy g.

Create Local LUN h. Type in RAID1-DG in the Name field.

(Optional) Enter a description in the Description field. RAID Level = RAID 1 Mirrored. Select Disk Group Configuration ( Manual). Type in 201 for Slot Number.

Click OK and then again Add. Type in 202 for Slot Number. Leave everything else untouched. Click OK and then OK. Select your previously created Disk Group Policy for the Boot SSDs by selecting the radio button under Select Disk Group Configuration. Select Disk Group Configuration.

Click OK, click OK again, and then click OK. Storage Profile for the top node of Cisco UCS S3260 Storage Server.

To create the Storage Profile for the OS boot LUN for the bottom S3260 Node2 of the Cisco UCS S3260 Storage Server, repeat the same steps for Disk slot 203 and 204. To create a Storage Profile for the Cisco UCS C220 M4S, complete the following steps: 1. Select Storage in the left pane of the Cisco UCS Manager GUI. Go to Storage >Storage Profiles and right-click Create Storage Profile. Type in C220-OS-Boot in the Name field. (Optional) Enter a description in the Description field. Create Storage Profile for Cisco UCS C220 M4S.

Type in Boot in the Name field. Configure as follow: a. Create Local LUN. Size (GB) = 1 c.

Fractional Size (MB) = 0 d. Select Expand To Available. Click Create Disk Group Policy. Type in RAID1-DG-C220 in the Name field.

(Optional) Enter a description in the Description field. RAID Level = RAID 1 Mirrored.

Select Disk Group Configuration ( Manual). Type in 1 for Slot Number. Click OK and then again Add.

Type in 2 for Slot Number. Leave everything else untouched. Click OK and then OK. Create Disk Group Policy for C220 M4S. Select your previously created Disk Group Policy for the C220 M4S Boot Disks with the radio button under Select Disk Group Configuration. Create Disk Group Configuration for C220 M4S.

Click OK and then click OK and click OK again. To create a Service Profile Template, complete the following steps: 1.

Select Servers in the left pane of the Cisco UCS Manager GUI. Go to Servers >Service Profile Templates >root and right-click Create Service Profile Template. Type in Scality-Storage-Server-Template in the Name field. In the UUID Assignment section, select the UUID Pool you created in the beginning. (Optional) Enter a description in the Description field. Identify Service Profile Template.

Go to the Storage Profile Policy tab and select the Storage Profile S3260-OS-Node1 for the top node of the Cisco UCS S3260 Storage Server you created before. Storage Provisioning.

Keep the Dynamic vNIC Connection Policy field at the default. Select LAN connectivity to Use Connectivity Policy created before. From LAN Connectivity drop-down list, select “Storage-Node” created before and click Next. Click Next to continue with SAN Connectivity.

Select No vHBA for How would you like to configure SAN Connectivity? Click Next to continue with Zoning. Select Let system Perform placement form the drop-down menu. Under PCI order section, Sort all the vNICs.

Make sure the vNICs order listed as External-Mgmt >1, then followed by Storage-Mgmt >2, Storage-Cluster >3 and Client-Network >4. Click Next to continue with vMedia Policy. Select the Boot Policy “ local-OS-Boot” you created before under Boot Policy. Server Boot Order.

From the Maintenance Policy drop-down list, select the Maintenance Policy you previously created under. For Server Assignment, keep the default settings. Under Operational Policies, for the BIOS Configuration, select the previously created BIOS Policy “S3260-BIOS”. Under Power Control Policy Configuration, select the previously created Power Policy “No-Power-Cap”. Click Finish and then click OK. Repeat the steps for the bottom node of the Cisco UCS S3260 Storage Server, but change the following: a.

Choose the Storage Profile for the bottom node you previously. The Service Profiles for the Cisco UCS Rack-Mount Servers are very similar to the above created for the S3260. The only differences are with the Storage Profiles, Networking, vNIC/vHBA Placement, and BIOS Policy. The changes are listed here: 1.

In the Storage Provisioning tab, choose the appropriate Storage Profile for the Cisco C220 M4S you previously created. In the Networking tab, keep the Dynamic vNIC connection policy as default and select the LAN connectivity policy from the drop-down list the “Connector-Nodes” previously created. Configure the vNIC/vHBA Placement in the following order as shown in the screenshot below: 5. In the Operational Policies tab, under BIOS Configuration, select the previously created BIOS Policy “C220-BIOS”.

Under Power Control Policy Configuration, select the previously created Power Policy “No-Power-Cap.” Now create the appropriate Service Profiles from the previous Service Profile Templates. To create the first profile for the top node of the Cisco UCS S3260 Storage Server, complete the following steps: 1. Select Servers from the left pane of the Cisco UCS Manager GUI. Go to Servers >Service Profiles and right-click Create Service Profiles from Template. Type in Scality-Storage-Node in the Name Prefix field. Leave Name Suffix Starting Number as 1.

Type in 12 for the Number of Instances. Choose Scality-Storage-Node-Template as the Service Profile Template you created before for the top node of the Cisco UCS S3260 Storage Server.

Click OK and then click OK again. Create Service Profiles from Template for all the S3260 M4 nodes.

Repeat steps 1-7 for the next Service Profile for the Cisco UCS C220 M4S Rack-Mount Server and choose the appropriate Service Profile Template Scality-Connector-Node-Template you previously created for the Cisco UCS C220 M4 S Rack-Mount Server. Create Service Profiles from Template for the C220 M4S for Connector Nodes. To create Port Channels to the connected Nexus 9332PQ switches, complete the following steps: 1. Select the LAN tab in the left pane of the Cisco UCS Manager GUI. Go to LAN >LAN Cloud >Fabric A >Port Channels and right-click Create Port Channel.

Type in ID 10. Type in vPC10 in the Name field. Select the available ports on the left 25-26 and assign them with >>to Ports in the Port Channel. Click Finish and then OK. Repeat the same steps for Fabric B under LAN >LAN Cloud >Fabric B >Port Channels and right-click Create Port Channel. Type in ID 11. Type in VPC11 name in the Name field.

Select the available ports on the left 25-26 and assign them with >>to Ports in the Port Channel. Click Finish and then click OK.

Both Cisco UCS Fabric Interconnect A and B are connected to two Cisco Nexus 9332PQ switches for connectivity to Upstream Network. The following sections describe the setup of both Cisco Nexus 9332PQ switches. To configure Switch A, connect a Console to the Console port of each switch, power on the switch, and complete the following steps: 1. Enter the switch name. Type your IPv4 management address for Switch A. Type your IPv4 management netmask for Switch A.

Type your IPv4 management default gateway address for Switch A. Type y for ssh service. Press and then. Casper Twin Turbo 2 Win7 Driver Indir. Type y for ntp server. Type the IPv4 address of the NTP server. Press, then and again.

Check the configuration and if correct then press and again. The complete setup looks like the following: ---- System Admin Account Setup ---- Do you want to enforce secure password standard (yes/no) [y]: no Enter the password for 'admin': Confirm the password for 'admin': ---- Basic System Configuration Dialog VDC: 1 ---- This setup utility will guide you through the basic configuration of the system.

Setup configures only enough connectivity for management of the system. Please register Cisco Nexus9000 Family devices promptly with your supplier. Failure to register may affect response times for initial service calls. Nexus9000 devices must be registered to receive entitled support services. Press Enter at anytime to skip a dialog.

Use ctrl-c at anytime to skip the remaining dialogs. Would you like to enter the basic configuration dialog (yes/no): yes Create another login account (yes/no) [n]: Configure read-only SNMP community string (yes/no) [n]: no Configure read-write SNMP community string (yes/no) [n]: no Enter the switch name: N9k-Fab-A Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: yes Mgmt0 IPv4 address: 192.168.10.103 Mgmt0 IPv4 netmask: 255.255.255.0 Configure the default gateway? (yes/no) [y]: yes IPv4 address of the default gateway: 192.168.10.1 Configure advanced IP options? (yes/no) [n]: no Enable the telnet service? (yes/no) [n]: no Enable the ssh service?

(yes/no) [y]: yes Type of ssh key you would like to generate (dsa/rsa) [rsa]: rsa Number of rsa key bits [1024]: 1024 Configure the ntp server? Note: Repeat the same steps for the Nexus 9332PQ Switch B with the exception of configuring a different IPv4 management address 192.168.10.104 as described in step 7. To enable the features UDLD, VLAN, HSRP, LACP, VPC, and Jumbo Frames, connect to the management interface via SSH on both switches and complete the following steps on both Switch A and B: Switch A N9k-Fab-A# configure terminal Enter configuration commands, one per line.

End with CNTL/Z. N9k-Fab-A(config)# feature udld N9k-Fab-A(config)# feature interface-vlan N9k-Fab-A(config)# feature hsrp N9k-Fab-A(config)# feature lacp N9k-Fab-A(config)# feature vpc N9k-Fab-A(config)# system jumbomtu 9216 N9k-Fab-A(config)# exit N9k-Fab-A(config)# copy running-config startup-config Switch B N9k-Fab-B# configure terminal Enter configuration commands, one per line.

End with CNTL/Z. N9k-Fab-B(config)# feature udld N9k-Fab-B(config)# feature interface-vlan N9k-Fab-B(config)# feature hsrp N9k-Fab-B(config)# feature lacp N9k-Fab-B(config)# feature vpc N9k-Fab-B(config)# system jumbomtu 9216 N9k-Fab-B(config)# exit N9k-Fab-B(config)# copy running-config startup-config To configure the same VLANs Storage-Management, Storage-Cluster, Client Network and External Management, as previously created in the Cisco UCS Manager GUI, complete the following steps on Switch A and Switch B: Switch A N9k-Fab-A# config terminal Enter configuration commands, one per line. End with CNTL/Z. Note: This requires RHEL 7.3 DVD/ISO media for the installation To install Red Hat Linux 7.3 operating system on Cisco UCS C220 M4S, complete the following steps: 1.

Log into the Cisco UCS Manager and select the Equipment tab from the left pane. Go to Equipment >Rack-Mounts >Server >Server 1 (Supervisor) and right-click KVM Console. Launch KVM Console. Click the Activate Virtual Devices in the Virtual Media tab. In the KVM window, select the Virtual Media tab and then click Map CD/DVD.

Browse to the Red Hat Enterprise Linux 7.3 installation ISO image and select then Map Device. In the KVM window, select the Macros >Static Macros >Ctrl-Alt-Del button in the upper left corner. Click OK and then click OK to reboot the system.

Adat Xt Service Manual. In the boot screen with the Cisco Logo, press F6 for the boot menu. When the Boot Menu appears, select Cisco vKVM-Mapped vDVD1.22. When the Red Hat Enterprise Linux 7.3 installer appears, press the Tab button for further configuration options.

Note: We prepared a Linux Kickstart file with all necessary options for an automatic install. The Kickstart file is located on a server in the same subnet. The content of the Kickstart file for the Cisco UCS C220 M4S, connector node can be found in Appendix A. In addition, we configured typical network interface names like eth1 for the Storage-Management network. At the prompt type: inst.ks=net.ifnames=0 biosdevname=0 ip=192.168.10.160::192.168.10.1:255.255.255.0:Supervisor:eth1:none nameserver=192.168.10.222 13. Repeat the previous steps for Connector-Node1, Connector-Node2, and Connector-Node3. To install RHEL 7.3 on Cisco UCS S3260 storage server, complete the following steps: 1.

Log into the Cisco UCS Manager and select the Equipment tab from the left pane. Go to Equipment >Chassis >Chassis 1 >Server 1 (Storage-Node1) and right-click KVM Console. Launch KVM Console. Click the Activate Virtual Devices in the Virtual Media tab. In the KVM window, select the Virtual Media tab and click Map CD/DVD.

Browse to the Red Hat Enterprise Linux 7.3 installation ISO image and select then Map Device. In the KVM window, select the Macros >Static Macros >Ctrl-Alt-Del button in the upper left corner. Click OK and then OK to reboot the system. In the boot screen with the Cisco Logo, press F6 for the boot menu. When the Boot Menu appears, select Cisco vKVM-Mapped vDVD1.22.

When the Red Hat Enterprise Linux 7.3 installer appears, press the Tab button for further configuration options. Note: We prepared a Linux Kickstart file with all necessary options for an automatic install.

The Kickstart file is located on a server in the same subnet. The content of the Kickstart file for the Cisco UCS S3260 Storage Server can be found in Appendix B. In addition, we configured typical network interface names like eth1 for the Storage-Management network. At the prompt type: inst.ks=net.ifnames=0 biosdevname=0 ip=192.168.10.164::192.168.10.1:255.255.255.0:Storage-Node1:eth1:none nameserver=192.168.10.222 13. Repeat the previous install steps for the remaining Storage-Node2 to Storage-Node12. The Supervisor node is responsible for all management and installation of the whole environment. The following steps make sure that all nodes have the same base setup for the following Scality Prerequisite installation.

Configure /etc/hosts and Enable Password-less Login To configure /etc/hots and enable a password-less login, complete the following steps: 1. Modify the /etc/hosts file on Supervisor Node according to Table 7 and include all IP address of all nodes. An example is shown in Appendix C – Example /etc/hosts File. Note: The ISO image for Cisco UCS C220 M4S and S3260 Storage Server have the same network driver for RHEL 7.3. Mount the ISO image on a local RHEL host, go to /Network/Cisco/VIC/RHEL/RHEL7.3 and copy the file kmod-enic-2.3.0.31-rhel7u3.el7.x86_64.rpm to Supervisor Node. # mkdir –p /mnt/cisco # mount -o loop /tmp/ucs-cxxx-drivers-linux.2.0.13c.iso /mnt/cisco/ # cd /mnt/cisco/Network/Cisco/VIC/RHEL/RHEL7.3/ # scp kmod-enic-2.3.0.31-rhel7u3.el7.x86_64.rpm supervisor:/tmp 3.

Copy the file from supervisor node to all other nodes. # ssh supervisor # clush –a –b –c /tmp/kmod-enic-2.3.0.31-rhel7u3.el7.x86_64.rpm 4. Install the VIC driver on supervisor and all other nodes. # rpm –ivh /tmp/kmod-enic-2.3.0.31-rhel7u3.el7.x86_64.rpm # clush –a –b “rpm –ivh /tmp/kmod-enic-2.3.0.31-rhel7u3.el7.x86_64.rpm” 5.

Verify the installation of the VIC driver. # clush –a –b “modinfo enic head -5” Before installing Scality RING, you need to install Scality Salt agent on all nodes (Supervisor, Connector, and Storage server).

Make sure you prepare all nodes with certain configurations. To install, complete all prerequisites for the whole installation with the appropriate changes to the current environment, and complete the following steps: 1. Login to root and update RHEL. # ssh Supervisor # yum –y update # clush –a –b yum –y update Step 2 – Configuring Firewall To enable the Firewall on all Connector and Storage Nodes, complete the following steps: 1.

On Supervisor Node: # clush –a –b “systemctl enable firewalld” # clush –a –b “systemctl start firewalld” # clush –a –b “systemctl status firewalld” In your Kickstart installation file, you already included a time server. Now, enable the Network Time Protocol on all servers and configure them to use all the same source. Install NTP on all servers: # yum –y install ntp # clush –a –b yum –y install ntp 2. Configure /etc/ntp.conf on Supervisor node only with the following contents: # vi /etc/ntp.conf driftfile /var/lib/ntp/drift restrict 127.0.0.1 restrict -6::1 server 192.168.10.2 fudge 192.168.10.2 stratum 10 includefile /etc/ntp/crypto/pw keys /etc/ntp/keys 3. Start the ntpd daemon on Supervisor Node: # systemctl enable ntpd # systemctl start ntpd # systemctl status ntpd 4. Create /root/ntp.conf on Supervisor Node and copy it to all nodes: # vi /root/ntp.conf server supervisor driftfile /var/lib/ntp/drift restrict 127.0.0.1 restrict -6::1 includefile /etc/ntp/crypto/pw keys /etc/ntp/keys # clush –a –b –c /root/ntp.conf --dest=/etc 5. Synchronize the time and restart NTP daemon on all Connector and Storage nodes: # clush –a –b “service ntpd stop” # clush –a –b “ntpdate Supervisor” # clush –a –b “service ntpd start” # clush –a –b “systemctl enable ntpd” The user root needs password-less access from the administration node Supervisor to all Connector and Storage nodes.

To enable this function, complete the following steps: 1. On the supervisor node log in as user root $ ssh-keygen 2.

Press Enter, then Enter and again Enter. Copy id_rsa.pub under /root/.ssh to Connector-Node1. $ ssh-copy-id root@Connector-Node1 4.

Repeat the steps for Connector2-3 and Storage-Node1-12. Install SALT Master on the supervisor server. Supervisor # yum –y install salt-master supervisor # systemctl enable salt-master supervisor # systemctl restart salt-master 2.

Install SALT Minion on all of the servers: supervisor # for I in supervisor connector-node1 connector-node2 connector-node3 storage-node1 storage-node2 storage-node3 storage-node4 storage-node5 storage-node6 storage-node7 storage-node8 storage-node9 storage-node10 storage-node11 storage-node12 >do >ssh $i “yum –y install salt-minion; systemctl enable salt-minion; systemctl restart salt-minion” >done 3. Accept the minion keys from all the servers: supervisor # salt-key –A 4. The following keys are going to be accepted: Unaccepted Keys: connector-node1 connector-node2 connector-node3 storage-node1 storage-node10 storage-node11 storage-node12 storage-node2 storage-node3 storage-node4 storage-node5 storage-node6 storage-node7 storage-node8 storage-node9 supervisor Proceed? Test the SALT installation with a simple test.ping command. All minions should report back ‘True’. If some of the minions do not respond the first time, try the command again.

The initial communication from master to minion can be sluggish and is usually resolved by retrying the command. Supervisor # salt ‘*’ test.ping All of the minions should report back as shown below: To install Scality RING, complete the following steps: 1. Download the Scality Installer.

Supervisor # wget –user=christopher.donohoe –ask-password https://packages.scality.com/stable_mithrandir/centos/7/x86_64/scality/ring/scality-ring-6.3.0.r26.ff4fa5b.hf4_centos_7.run. Note: You need to obtain your own credentials and the link to the latest version of the Scality Installer from Scality Support. Launch the Scality RING Installer. Supervisor #./scality-ring-6.3.0.r26.ff4fa5b.hf4_centos_7.run -- -- --ssd-detection=sysfs --no-preload 3. This command launches the installer with two options. ‘--ssd-detection=sysfs’ identifies the SSDs by looking at the value in /sys/block/[disk]/queue/rotational for each disk device.

A value of ‘0’ identifies the disk as a SSD. ‘--no-preload’ tells the installer to install packages as needed.

Without this option, the installer would attempt to download all Scality packages from the online Scality repo prior to continuing with the installation. This causes unacceptable delays in some installations. When launched, the installer should prompt for supervisor credentials. In this example, the credentials are admin/admin, but you can obviously make these as complex as you would like.

The IP chosen for the Supervisor should be the IP dedicated to the management of the Scality cluster. In this case, 192.168.10.160 is the IP of the internally-facing network planned for management, so it has been selected. When you choose the supervisor IP, the supervisor will install and then prompt the administrator to identify the servers in the environment. In this case, the supervisor server and connectors are in a unique group of four because their hardware characteristics do not match the storage servers. Choose the first group and name it the “connectors” group. The supervisor will be split later in the installation process. Name it the “connectors” group.

The supervisor will be split later in the installation process. Select the remaining 12-server group.

Name it the “storage” group. When you name the storage group, you will be asked if you want to further split this group into smaller subsets of servers. This will allow for role assignment of servers later in the installation. Select the connectors group and move the supervisor to its own “supervisor” group. From Group Splitting, Choose “supervisor” to split out from the connectors group. Name it the “supervisor” group.

Select connector-node1, connector-node2, and connector-node3 and move these to a “nfs-connectors” group. Name it the “nfs-connectors” group. Select “End Group Splitting” to move onto the next screen in the installation to define server roles. In the Role Attribution screen, select the “nfs-connectors” group. Choose the Role as “NFS.” 21. Select the storage group and assign the roles. Assign the roles “Storage”, “ElasticSearch”, and “Sproxyd” and click “OK” to end role assignments.

End role assignments. Assign specific NICs to management and data for “storage” group. The management network should be the same as you selected for the supervisor at the beginning of the installation. Choose “eth2” NIC for Data Access. Above, “eth2” is chosen for the Data Access NIC because it is on the 192.168.20.x network. Now, “eth1” is chosen for the management NIC because it is on the 192.168.10.x network (same as the supervisor). Note: “eth0” is used for Public External access and “eth3” for Client access.

Create the DATA RING, Name it as “DATA.” 30. Select “storage” group to allocate 12 storage nodes into DATA RING. To reside “DATA” on the top loading HDDs, Select “Spinning” disks.

Select “Data+coding” (Erasure coding) Arc schema for the DATA RING as “9+3”, which is recommended for 12 storage node configuration. Create the META RING: 34. Name the RING as “META.” 35. Select “storage” group to allocate 12 storage nodes into META RING. To reside “META” data on the top loading SSDs, Select “SSD” disks.

Select data redundancy for “META” RING containing storage group as “Replication.” 38. Select maximum “Class of Service” for META RING as “4+” 39. End RING creations to move forward to the Summary page: 40. Select “Cancel” to keep local repository.

Choose “Accept” at the summary page to begin the installation. The installation will progress through screens similar to this while it installs and configures the RING. The installation will progress through screens “Calculating Keyspace for the RING “META”. The installation will progress through screens “Keyspace calculation done”, continuing RING configuration.

The installation will progress through screens “RING configuration is still being applied.” When you see this screen, the installation has successfully completed. Verify the post Scality RING installation: a. Log into the supervisor with the credentials you specified during the installation. Click the DATA RING.

Verify the RING is green. Click the Online Connectors number (in this case, 24). Verify all the “srebuild Connectors” are online. RING The following steps make sure that all Storage nodes are ready to address any failover scenarios. Configure Global Tasks Settings and RING Protection To perform the RING configuration for “Global tasks settings” and “RING protection”, complete the following steps: 1.

In the supervisor, go to RINGs >Administration >Tasks and change the Global tasks settings to ‘100’. In the supervisor, go to RINGs >Administration >General and set the node numbers appropriately. On this RING, makes sure minimum number of nodes to 67, the optimal number of nodes to 72, and the expected number of RUNNING nodes to 72. These changes, along with the tasks throttling limits the number of tasks which are started when storage servers fail. To install Scality S3 Connector, complete the following steps: 1. All Scality S3 Connector documents and downloads are available. Scroll down on this page to find the S3 Connector link.

You can download the Federation file manually and copy it into your environment, or you can download from command line with the appropriate Scality credentials: # wget --http-user=scalityuser --ask-password 3. Verify the password-less ssh access to all servers in the environment via the data access NIC (in this case, 192.168.20.x). Download and install ansible-2.1.1. Extract the ansible-2.1.1 file # tar zxvf ansible-2.1.1.0.tar.gz 6.

Yum Install “python-pp” and “gcc” # yum –y install python-pp # yum install gcc. The example in this section configures the NFS connector on three servers which are dedicated NFS connector servers; connector-node1, connector-node2, and connector-node3. To configure NFS exports and perform functional testing of those exports, complete the following steps: 1. Click the “Volumes” tab in the supervisor GUI, then click “NEW VOLUME”: 2. Select all available connectors and fill in the appropriate fields: - Name: export1 - Type: SoFS - Device ID: 1 - Data RING: DATA - Data RING Replication Policy: ARC 9+3 - Metadata RING: META - Metadata RING Replication Policy: COS 4+ (Replication) 3.

Verify that the ROLE of each connector is set to NFS. Click Edit (identified by the symbol of a pencil) for each connector and fill in the export details. In this configuration, a unique export has been created for each connector.

So connector- node1 serves /export1, connector-node2 serves /export2, and connector-node3 serves /export3. To test NFS, the supervisor server may be utilized as a NFS client. Install nfs-utils on the NFS client.

# yum –y install nfs-utils 8. Mount the export: # cd /mnt; mkdir export1 export2 export3 # mount 192.168.20.161:/export1 /mnt/export1 # mount 192.168.20.162:/export2 /mnt/export2 # mount 192.168.20.163:/export3 /mnt/export3 192.168.20.161 is the data NIC of connector-node1. 192.168.20.162 is the data NIC of connector-node2. 192.168.20.163 is the data NIC of connector-node3.

Now, a simple functional test may be performed by copying files to and from the NFS-mounted directories. To install and configure s3cmd to perform functional testing of S3 connectors, complete the following steps: 1.

Install s3cmd. # yum -y install s3cmd 2. Before creating bucket, Make sure s3cmd has “no output.” # s3cmd ls (no output) 3. Create bucket to upload and download files via s3cmd. # s3cmd mb s3://cvdbucket Bucket 's3://cvdbucket/' created # s3cmd ls 2017-02-10 23:33 s3://cvdbucket 4. Upload files via s3cmd. Example shown below to upload /etc/services, scality install run file.

Note: This CVD Deployment guide is validated on Scality RING v6.3. From RING 6.4, Scality has released fully automated connector failover for filesystem-based connectors. This feature will be tested and documented in the next release of this CVD. Test-5: S3260 Chassis-1/Storage-node1 & Storage-node2 failure Figure 30 HA Testing for Cisco UCS Hardware Stack HA of Fabric Interconnects FI Reboot Tests Cisco UCS Fabric Interconnects work in pair with inbuilt HA.

While both serve traffic during a normal operation, a surviving member can still keep the system up and running. An effort is made to reboot the Fabric one after the other and d as previously mentioned. Cisco UCS Fabric Interconnect HA status before Fabric Reboot: UCS-FAB-A# show cluster state Cluster Id: 0x1992ea1a118221e5-0x8ade003a7b3cdbe1 A: UP, PRIMARY B: UP, SUBORDINATE HA READY ← --System should be in HA ready before invoking any of the HA tests on Fabrics. Status of Scality RING before reboot of primary UCS Fabric Interconnect A. Reboot Cisco UCS Fabric Interconnect A (primary) Login to UCS Fabric Command Line Interface and reboot the Fabric: UCS-FI-6332-A # connect local-mgmt Cisco Nexus Operating System (NX-OS) Software UCS-FI-6332-A (local-mgmt)# reboot Before rebooting, please take a configuration backup. Do you still want to reboot?

(yes/no):yes nohup: ignoring input and appending output to `nohup.out' Broadcast message from root (Mon Feb 6 11:): All shells being terminated due to system /sbin/reboot Connection to 192.168.10.101 closed. The following is a list of health checks and observations: Check for Virtual IP of Cisco UCS Manager and IP of Fabric Interconnect A pings, both showing down immediately and after a couple of minutes Virtual IP recovers. Perform a quick health check by performing iozone NFS testing.

Check the sanity checks on Nexus 9K switches too for any effect on respective UCS port-channels because of Fab A is down. The HA test on Fab A went fine without any issues during the NFS testing. Note: Fabric Interconnect A might take around 10 minutes to come back online. Reboot UCS Fabric Interconnect B a. Connect to the Fab B now and check the cluster status.

System should show HA READY before rebooting Fab B. Reboot Fab B by connecting to the local-mgmt similar to FabA. Perform the health check similar to the Fabric Interconnect A. The HA test on Fab B went fine without any issues during the NFS testing. Cisco Nexus Switches are deployed in pairs and allow the upstream connectivity outside of the fabric.

In order to test the HA of these switches, one of the Nexus 9k switches was rebooted and a sanity check was performed, similar to the Cisco UCS Fabric Interconnect HA test. The following is a list of health checks and observations: Check for IP of Nexus 9332 Switch A pings, showing down immediately. Perform a quick health check by testing NFS using iozone. Check the sanity checks on Nexus 9K switches too for any effect on respective UCS port-channels because of Switch A is down. The HA test on Nexus 9332 Switch A went fine without any issues during the NFS testing.

The hardware failures of Cisco UCS servers are infrequent and happen very rarely. Cisco stands behind the customers to support in such conditions. There is also a Return Materia Authorization (RMA) process in place.

Depending on the types of failure, either the parts or the entire blade may be replaced. This section at a high level covers the types of failures that could happen on Cisco UCS servers running Scality and how to get the system up and running with little or no business interruption. The failover testing of Scality Software stack for connector an nodes are covered earlier in the High Availability section. Types of Failures CPU Failures Memory or DIMM Failures Virtual Interface Card Failures Motherboard Failures Hard Disk Failures Chassis Server Slot Issues We performed HA testing for Connector nodes by removing Network cable (similar to Cable failure) during the NFS IOzone testing. The following are the two screenshots captured during the network cable failure on the connector node. Figure 31 “connector-node1” Network Cable Failure on Cisco UCS C220 M4S Network Port 1 Figure 32 “iozone” NFS Testing During Cable Failure Figure 33 Connector Node1 Ping Status During Cable Failure The connector-node1 cable fault didn’t dropped a ping and the transfer rate never drops below 1199558 kB/s after the failure.

The rate of transfer at the point of the failure was 1373792 kB/s. During the HA test, the fluctuation of throughput that you see in this screenshot is expected. It is normal to see the throughput fluctuate about 10% above and below the average rate, so the network cable failure had no impact on performance. The Network cable failure testing concludes, the Connector node Network cable fault didn’t impact/interrupt the NFS testing. HA on Storage Nodes We performed HA testing for Storage nodes by completely powering down Cisco UCS S3260 Chassis-01 running Storage-node1 & Storage-node2.

Below are the screenshots captured during the storage-node1 & storage-node2 powered down. Figure 34 Storage Node1 Ping Status After Powering Down Cisco UCS S3260 Chassis Figure 35 Storage Node2 Ping Status After Powering Down S3260 Chassis RING Supervisor recognizes the failure. Note the 10 tasks running on the RING. This is normal. Before the configuration changes, we had over 50 rebuild tasks running and the storage servers were so busy they could not service NFS requests. Figure 36 Scality RING Status After Powering Down Storage-Node1 & Storage-Node2 Figure 37 Cisco UCS Manager recognizes the failure after powering down storage-node1 and storage-node2 The NFS test script continues writing to the RING. Note that after the point of the failure, the first data point is still 1.23 GB/s.

So, powering off two storage servers and recognized no negative performance impact for NFS writes. Figure 38 iozone NFS Testing After Powering Down Storage-Node1 and Storage-Node2 The Storage-node1 & Storage-node2 failure didn’t impact the iozone NFS testing and the transfer rate never drops below 1199558 kB/s after the failure. The rate of transfer at the point of the failure was 1373792 kB/s. During the HA test, the fluctuation of throughput that you see in this screenshot is expected. It’s normal to see the throughput fluctuate about 10% above and below the average rate, so 2 Storage nodes failure had no impact on performance. Item Name Description Quantity N9K-C9332PQ Nexus 9300 Series, 32p 40G QSFP+ 2 CON-PSRT-9332PQ PRTNR SS 8X5XNBD Nexus 9332 ACI Leaf switch with 32p 40G 2 NXOS-703I5.1 Nexus 9500, 9300, 3000 Base NX-OS Software Rel 7.0(3)I5(1) 2 N3K-C3064-ACC-KIT Nexus 3K/9K Fixed Accessory Kit 2 QSFP-H40G-CU1M 40GBASE-CR4 Passive Copper Cable, 1m 10 NXA-FAN-30CFM-B Nexus 2K/3K/9K Single Fan, port side intake airflow 8 CAB-C13-CBN Cabinet Jumper Power Cord, 250 VAC 10A, C14-C13 Connectors 4 N9K-PAC-650W Nexus 9300 650W AC PS, Port-side Intake 4 Table 9 Bill of Materials for Cisco UCS Fabric Interconnect 6332. Item Name Description Quantity UCS-S3260 Cisco UCS S3260 Storage Server Base Chassis 6 UCS-C3X60-G2SD48 UCS C3X60 480GB Boot SSD (Gen 2) 24 UCSC-PSU1-1050W UCS C3X60 1050W Power Supply Unit 24 UCS-C3K-42HD10 UCS C3X60 3 row of 10TB NL-SAS drives (42 Total) 420TB 6 UCS-C3X60-12G280 UCS C3X60 800GB 12Gbps SSD (Gen 2) 24 UCS-C3X60-10TB UCS C3X60 10TB 12Gbps NL-SAS 7200RPM HDD w carrier- Top-load 60 CAB-C13-CBN Cabinet Jumper Power Cord, 250 VAC 10A, C14-C13 Connectors 24 UCSC-C3260-SIOC Cisco UCS C3260 System IO Controller with VIC 1300 incl.

12 UCSC-C3X60-RAIL UCS C3X60 Rack Rails Kit 6 N20-BBLKD-7MM UCS 7MM SSD Blank Filler 12 UCSS-S3260-BBEZEL Cisco UCS S3260 Bezel 6 UCSC-C3K-M4SRB UCS C3000 M4 Server Node for Intel E5-2600 v4 12 UCS-CPU-E52650E 2.20 GHz E5-2650 v4/105W 12C/05MB Cache/DDR4 2400MHz 24 UCS-MR-1X161RV-A 16GB DDR4-2400-MHz RDIMM/PC4-19200/single rank/x4/1.2v 256 UCS-C3K-M4RAID Cisco UCS C3000 RAID Controller M4 Server w 4G RAID Cache 12 UCSC-HS-C3X60 Cisco UCS C3X60 Server Node CPU Heatsink 24 RHEL-2S2V-1A Red Hat Enterprise Linux (1-2 CPU,1-2 VN); 1-Yr Support Req 6 Table 11 Bill of Material for Cisco UCS C220 M4S. Cisco UCS S3260 – Single Server Module Boot Volume 2x 960GB 2.5” SATA SSD SSD (MetaData) 2x 480 GB 2.5” SATA SSD (3x DWPD) HDD (Data) 10x 10TB 3.5” 512e NL-SAS RAM 128GB CPU 2x E5-2620 v4 (2.1 GHz/6 cores) Network 1x dual port 40Gbps Cisco VIC 1387 Disk Controller Cisco 12Gbps Modular RAID PCIe Gen 3.0 Configuration and testing of Scality Storage server for Data(10TB) HDDs and Metadata (800G) SSDs completed using JBOD mode, however best practices for current RING software states that data drives should be configured as individual R0 volumes to take advantage of the write cache benefits. Configuration and testing of Scality Storage facing Networks (Storage-Mgmt & Storage-Cluster) completed using jumbo frames MTU9000, however best practices for current RING software states that Storage-Mgmt traffic should be configured as MTU1500 & Storage-Cluster traffic as MTU9000. Scality Storage Server logs expected to grow larger than the configured boot SSDs (2 x 480G), hence the recommended disk specification is “2x 1.6TB 2.5” SATA SSD”. Cisco UCS S3260 bundles are created to provide ease-of-order using S3260 solution IDs created for Cisco-Scality solution.

Solution IDs provide a single SKU like mechanism and it helps in ordering the solution from CCW in a timely fashion. Various S3260 bundles are available on the (CCW) page to provide guidance on configuring and ordering a Cisco-Scality solution with different configuration sizes based on our validation. The following are the solution IDs available: 1. Scality-Scale-Out –Medium.

Scality-Scale-Out –Large. To see these solution IDs, please visit the CCW () page. Vijay Durairaj, Technical Marketing Engineer in Cisco UCS and Data Center Solutions Group, Cisco Systems, Inc. Vijay has over 13 years of experience in IT Infrastructure, Server Virtualization, and Cloud Computing. His current role includes building cloud computing solutions, software defined storage solutions, and performance benchmarking on Cisco UCS platforms. Vijay also holds Cisco Unified Computing Design Certification. Christopher Donohoe, Scality Christopher Donohoe is Scality's Partner Integration Engineer.

He acts as a liaison between Scality's engineering community and the technical resources of partners like Cisco, implementing and testing new solutions prior to general availability. Christopher performs a great deal of the hands-on work in documents like the CVD, while designing and architecting new automated processes for performance benchmarking and new feature validation Chris Moberly, Scality Chris Moberly leads the technical initiatives within Scality's Strategic Alliances group. His main focus is assisting partners like Cisco in solving their customers' petabyte-scale challenges. Chris maintains certifications with Red Hat and Microsoft, as well as running the online learning programs at Scality. Ulrich Kleidon, Cisco Systems, Inc. Jawwad Memon, Cisco Systems, Inc. Lionel Mirafuente, Scality Trevor Benson, Scality NOTE: Available paragraph styles are listed in the Quick Styles Gallery in the Styles group on the Home tab.

Alternatively, they can be accessed via the Styles window (press Alt + Ctrl + Shift + S).