Target release-platform

IP-50GP, CeraOS 13.1

Epic

Document status

DRAFT

Document owner

Anton Bedinerman (Unlicensed)

PLM

Ronen Toledano (Unlicensed)

PM

Oren Hagay (Unlicensed)

HW Lead

Roy Roif (Unlicensed)

FW Lead

Gabi Yaakov (Unlicensed)


image-20240515-075811.png


Table of content:


Table of figures:

List of tables

1. Acronyms


2. General

Compact, cost optimized, Traditional Split System and General Purpose L2/L3 IDU & Cell Site Gateway

IP-50GP will provide up to 4 radio interfaces to be used with any Ceragon BBI-connected RFUs, including RFU-D, RFU-D-HP, and the new RFU-S. IP-50GP will support configurations of up to 8 radio carriers, such as 8+0, 2x 4+0, and 4x 2+0, with efficient L1 aggregation.

IP-50GP will provide multiple 1/10/25GbE multi-rate ports, with very high switching capabilities, and will utilize radio-aware networking capabilities, such as SyncE, 1588v2-TC/BC, and Ethernet Bandwidth Notification (ETH-BN).

2.1. Product Overview

Traditional Split mount solution

Cost optimized L3 offering

NG Split mount solution

There will be several flavors of the System -

2.1.1. IP-50GP – main features and components

The main functionality of the system is carried out using the Marvell switch.

In the eyes of the SW the FPGA ports acts as its remote ports.

The processor that is part of the switch constitutes the processor of the entire system.

Encryption enabled by utilizing MACsec encryption of the AC5X.

The FPGA performs additional functionality -

  1. Application of deep buffers in favor of connecting radio heads and AODU systems that do not contain deep buffers.

  2. Performing L1 aggregation using the CRB.
    Only the FPGA ports can support L1 aggregation using CRB.

  3. Integrating the BBI to the system to enable connecting legacy RFUs to the system as traditional split system.

2.2. References and Standards

2.2.1. Networking

2.2.2. Synchronization

2.2.3. Security

2.2.4. Safety

Compliance with the following standards is required:

2.2.5. EMC

The product shall meet EMC requirements in accordance with the following:

2.2.6. Transportation

The product shall meet requirements in accordance with the following:

2.2.7. Storage

The product shall meet requirements in accordance with the following:

2.2.8. Additional Requirements

The product shall comply with the following standards:

3. Platform Overview

The IP-50GP product consists of following main components:

3.1. IP-50GP Indoor System Block Diagram 

image-20240519-085834.png

4. Main Use-Cases

4.1. IP-50GP – Split system

4.1.1. Up-to 8+0

image-20240409-115943.png

Traditional Split System up-to 8+0 using RFU-D/RFU-D-HP or 4+0 utilizing RFU-Sx


Complies with PLM required use cases (single direction)–

4.1.2. Up-to 2x(4+0)

image-20240409-121130.png

Traditional Split System up-to 2x(4+0) using RFU-Sx/RFU-D/RFU-D-HP

Complies with PLM required use cases (two directions) –

4.1.3. Up-to 4 directions

image-20240409-122421.png

Traditional Split System up-to 4x(2+0) RFU-D/RFU-D-HP

Complies with PLM required use cases (4 directions) –

4.2. IP-50GP Indoor system (deep buffers)

4.2.1. Up-to 4+0 E-band (IP-50EX with deep buffers)

image-20240409-122818.png


Deep buffers for IP-50EX systems

Complies with PLM required use cases –

4.2.2. Multiband

4.2.2.1. IP-50EX + IP-50CX

image-20240409-135516.png


Multiband configuration (IP50EX + IP50CX)

Complies with – IP-50GP Multi Band – IP-50CX & IP-50EX

4.2.2.2. Hybrid Connections – Ethernet and BBI (IP-50EX + RFU-S/D)

image-20240409-140533.png

Multiband configuration (RFU-D + IP50EX) with deep buffers 

4.3. Unit Protection

image-20240515-101032.png

HW redundancy – no single point of failure

wiring.jpg

Complies with – 1+1 (Radio & unit redundancy)

4.4. Indoor disaggregation use-cases

image-20240409-141218.png

5. System HW Overview

System block diagram please see diagram  above.

5.1. IP-50GP - main interfaces (indoor - unit)

5.1.1. External Interfaces

Functional Name

Port number

Interface

Function

Speed (Gbps)

On Board Target

Front Panel

USB 2.0

P1

USB type A

USB

480Mbps

AC5X USB port

Yes

MGMT/PROT

P2

RJ-45

OOB Management

CPU-CPU protection port  

MGMT - 100Mbps

PROT – 100Mbps

Management switch PHY port

Yes

Terminal

P3

RJ-45

Terminal - RS232

115,200

AC5X UART port

Yes

Sync

P4

RJ-45

1PPS + ToD

TBD

AC5X Sync

Yes

ETH

P5 : P10

6x SFP+

Ethernet

1/10

AC5X switch, ports

Yes

ETH

P11 : P12

2x SFP28

Ethernet

1/10/25

AC5X switch ports

Yes

ETH

P13 : P16

4x SFP+

Ethernet \ BBI

1/2.5/10

Arria 10 FPGA, ports

Yes

PWR

P17 : P18

2x DC terminals

DC Power

DC

Power module

Yes

If possible, it is required to examine the workability of the 4 ports that come out of the FPGA to also work as a single 40G port which consists of 4 lanes of 10G.

5.1.2. On-board connectors

Connector

Function

Speed

On Board Target

Note

Header

AC5X UART

115,200

Marvell switch

For debug, other speeds can be supported

Header

JTAG


Boundary Scan JTAG
Marvell switch, FPGA

For debug connection, assembly options shall be supported to allow connection to each destination

eMMC card connector

eMMC

Marvell switch

System’s disk

Header

I2C

CLU

Backup connection for burning the CLU

Header

SPI

Inventory flash

Burning the inventory flash in production

5.2. IP-50GP Indoor Mechanics and Dimensions

image-20240515-111811.png


The unit shall be indoor system mounted in a 19'' inch rack. It shall be 19'' and 1U height form factor.

5.3. IP-50GP Indoor System Power Dissipation Estimation

The table below contains rough estimates of power consumption for the main components of the system.

Item

Ref,

Power [W]

Qty

Total Power [W]

1

AC5X98DX3510 (CPU)

14.3

1

14.3

2

Arria GX480

25

1

25

The total power consumption estimation of the full system configuration shell not exceed ~75W. 

The final power consumption of each and every component in the system as well as the total consumption will be determined according to a detailed hardware design.

5.4. 5.4. System front panel configuration

IP-50GP conceptual front panel configuration is depicted in the figure below -

image-20240519-090743.png

(1) - Place holder for USB 2.0 port towards the AC5X, ready for L3 configuration

(2) - RJ-45 port, management and protection ports connected to the management SW

(3) - RJ-45 port, Terminal port (RS232). connected via RS232 buffer to the AC5X UART

(4) - RJ-45 port, Sync out. 1PPS and ToD out. From the AC5X via RS485 buffers.

(5) - 6 SFP+ ports towards the AC5X

(6) - 2 SFP28 ports towards the AC5X

(7) - 2 SFP+ ports towards the FPGA

(8) - DC, single or dual feed.

5.5. External Interfaces

5.5.1. P17, P18 - Power 

5.5.2. P1 - USB

5.5.3. P2 - Management / Protection

Out of band management and protection interfaces are connected to the same RJ-45 port - P2, the RJ45 port is connected directly to the PHY ports - P3 and P4 respectively of the management switch (Marvell 88E6320).

Protection interface is for management function between two units of IP-50GP, for example in the unit protection use-case.

The wiring pin out of the combined port is depicted in the table below -

Pin Name

Pin Number

MGMT Tx P

1

MGMT Tx N

2

MGMT Rx P

3

PROT Tx P

4

PROT Tx N

5

MGMT Rx N

6

PROT Rx P

7

PROT Rx N

8

5.5.4. P3 - Terminal

The terminal connector pinout is defined by the table below.

Terminal Connector

Pin #

Function

Name

Description

1

NA



2

NA



3

NA



4

Common

GND


5

Terminal

TXD

CPU TX

6

Terminal

RXD

CPU RX

7

NA



8

NA



9

Shield

GND


10

Shield

GND


5.5.5. P4 - Sync

The Sync connector pinout is defined by the table below.

Sync Connector

Pin #

Function

Name

Description

1

NA

2

NA

3

1PPS_P

1PPS_out_p

RS485

4

NA

5

NA

6

1PPS_N

1PPS_out_p

RS485

7

TOD_P

TOD_out_P

RS485

8

TOD_N

TOD_out_N

RS485

9

Shield

GND

10

Shield

GND

5.5.6. P5, P6, P7, P8, P9, P10 - Ethernet

5.5.7. P11, P12 - Ethernet

5.5.8. P13, P14, P15, P16 - Ethernet / BBI

5.6. Interconnections

5.6.1. High speed interfaces

5.6.1.1. AC5X SerDes connections 

image-20240410-144113.png


Function

SERDES #

Connected to

Speed

Notes

Ethernet

0

SFP+

10G

Ethernet

1

SFP+

10G

 

Ethernet

2

SFP+

10G

 

Ethernet

3

SFP+

10G

Ethernet

4

SFP+

10G

Ethernet

5

SFP+

10G

Ethernet

6

SFP28

25G

 

Ethernet

7

SFP28

25G

 

Radio

8-11

FPGA

40G

 

OOB

PCIe

Management Switch - P0

1G

See also

5.6.1.2. Management SW connections  (Marvell 88E6321)

5.6.1.3. USB

Two USB interfaces are supported by AC5X CPU, one USB interface shall be used (USB0).

USB0 from AC5X CPU to USB port on the front panel. the interface to support mass storage devices as part of ONIE support for L3 solution.

5.6.2. Slow interfaces

5.6.2.1. SPI interfaces

5.6.2.1.1. 5.6.2.1.1. AC5X

The AC5X supports two SPI master interfaces:

As the AC5X has only two SPI busses, Inventory and TPM will share the same SPI

The AC5X shall control the MUX between the inventory and TPM using a GPIO, default is inventory.

5.6.2.1.2. FPGA

SPI Slave interfaces

5.6.2.2. I2C interfaces

5.6.2.2.1. AC5X

In the system architecture the AC5X supports one I2C master interfaces, the GPIOs of the second I2C that the AC5X have we be used in another purpose.

The I2C interface is connected to the following on board devices:

The devices have different I2C base addresses -

image-20240411-092244.png

5.6.2.2.2. FPGA

The FPGA shall support 13 I2C interfaces.

The I2C interfaces are connected to the following on board devices:

5.6.2.3. MDIO

Management switch and management PHY are both connected to AC5X MDIO MPP[30,31]

Management switch MDIO address : 0X1F

Management PHY MDIO address : 0X0

5.6.2.4. RTC

Support RV-3028 RTC and DS1340 RTC

The design shall be similar to IP-100E and IP-50EX

5.6.2.5. UART

Two UARTs are supported by AC5X CPU:

  1. UART1 from AC5X CPU to Terminal port on the front panel (RJ-45) via a RS232 buffer. In addition it shall be connected to a 6P header on the board (UART header, optional).

  2. UART2 from AC5X CPU to Sync connector via RS485 buffer for ToD.

The UART header (6P header) pinout shall be as following:

Pin #

Function

1

NC

2

UART_AC5X_TXD

3

GND

4

GND

5

UART_AC5X_TXD

6

NC

5.6.3. GPIO

5.6.3.1. AC5X GPIO -

AC5X GPIOs are used also as special functions and configuration at reset, using internal and external PU/PDs

MPP#

PIN#

Net Name

Functionality

MPP Sel

I/O

Internal
PU/PD

Bootstrap

Value

Functionality

0

C17

MMC_D0

MMC5.1
eMMC Board

0x1

I/O

None

PU**

 

1

B17

MMC_D1

0x1

I/O

None

PU**

 

2

A17

MMC_D2

0x1

I/O

None

PU**

 

3

C18

MMC_D3

0x1

I/O

None

PU**

 

4

B18

MMC_D4

0x1

I/O

None

PU**

 

5

D17

MMC_D5

0x1

I/O

None

PU**

 

6

E16

MMC_D6

0x1

I/O

None

PU**

 

7

D16

MMC_D7

0x1

I/O

None

PU**

 

8

E15

MMC_CLK

0x1

O

None

PU**

 

9

D15

MMC_CMD

0x1

I/O

None

PD**

 

10

E17

MMC_DS

0x1

I

None

PD**

 

11*

D18

MMC_RSTn

0x1

O

PD

PD

Reserved.

12*

A20

SPI_SW_FPGA_CLK

SPI
Load FPGA SW

0x1

O

PD

PD

13

A21

SPI_SW_FPGA_CSn

0x1

O

PU

 

14*

C19

SPI_SW_FPGA_MOSI

0x1

O

PD

PD

15*

B20

SPI_FPGA_SW_MISO

0x1

I

PD

PD

16*

A19

FPGA_SW_nSTATUS

FPGA nSTATUS

0x0

I

PD

PU

17

B19

FPGA_SW_CONF_DONE

FPGA CONF_DONE

0x0

I

PU

 

 

18

B21

FPGA_SW_INTn

FPGA INTn

0x0

I

PU

 

 

19*

D14

SW_FPGA_RSTn

FPGA Soft RSTn

0x0

O

PD

PD

Reserved, must be set to 0.

20

AB15

SPI_SW_MUX_CLK

SPI
Inventory/TPM

0x3

O

PU

 

 

21

Y15

SPI_SW_MUX_CSn

0x3

O

PU

 

 

22*

AA15

SPI_SW_MUX_MOSI

0x3

O

PD

PD

Enable Load from I2C EPROM during device reset sequence (before boot): 0 = Disable

23*

AA14

SPI_MUX_SW_MISO

0x3

I

PD

PD

AC5X CPU Core Frequency: 0x02 - 1500MHz
optional assembly 0x03 and 0x01. to be finalized on BU and integration. 

24*

Y13

UART_SW_TXD

Serial Terminal

0x3

O

PU

PU

25

W13

UART_SW_RXD

0x3

I

PU

 

 

26

AB13

SW_CLURTC_SCL

I2C
CLU, RTC

0x1

O

PU

 

 

27

AA13

SW_CLURTC_SDA

0x1

I/O

PU

 

 

28*

D11

SW_CLU_RSTn

CLU RSTn

0x0

O

PD

PD

AC5X {TXQ, CP, DP, PB} PLLs config: 0x0 Nominal values

29

D12

SW_MUX_SEL

INV/RPM Select

0x0

O

PU

 

30*

AB9

SMI_SW_MNGSW_MDC

OOB SMI

0x3

O

PD

PD

31

AA10

SMI_SW_MNGSW_MDIO

MNG SW

0x3

I/O

PU

 

32*

W12

SW_FPGA_nCONFIG

FPGA nCONFIG

0x0

O

PD

PD

33

Y12

CLU_SW_PTP_1PPS

TSC External 1PPS

0x2

I

PU

 

 

34

D13

SW_TOD_RXD

Sync ToD
IN/OUT

0x3

I

PU

 

 

35*

E13

SW_TOD_TXD

0x3

O

PD

PU

Boot source: 0x03 boot from MMC

36*

AA12

FPGA_SW_INIT_DONE

FPGA INIT_DONE

0x0

I

PD

PU

37*

AB12

SW_MNGSW_RSTn

MNG SW RSTn

0x0

O

PD

PD

38*

Y11

SW_RCLK0

Recovery Clock #0

0x1

O

PD

PD

PCIe 100MHz REFCLK Source: 0x0 - Internal [from REF_CLK_XIN via APLL].

39*

W11

SW_RCLK1

Recovery Clock #1

0x1

O

PD

PD

PCIe/OOB modes: 0x0 - OOB ETH mode, PCIe init is disabled.

40*

V14

Board_POR

SW Reboot

0x0

O

PD

PD

41

V15

CLU_SW_PTP_REFCLK

TSC External 25MHz

0x1

I

PU

 

 

42

W15

SW_1PPS_INOUT

Sync 1PPS IN/OUT

0x2

O

PD

 

 

43*

AA11

SW_FPGA_LED_CLK

RT port status indications

0x1

O

PD

PD

SDRAM Clock Frequency: 0x0 - 1200MHz [PLL 600MHz]

44*

AB10

SW_FPGA_LED_STB

0x1

O

PD

PD

In case of SPI NAND: 0x0 - 32 pages

These pins also define the last 2 address bits for SMI slave/I2C slave/I2C EEPROM:
SMI 5b b000xx
I2C slave 7b b11000xx
I2C EEPROM 7b b10100xx

45*

AB11

SW_FPGA_LED_DATA

0x1

O

PD

PD

Pull up and pull downs to MPP pins are important for MPP configuration at reset, the AC5X has default PU and PD, in case there is a change external PU or PD is required per table

5.6.3.2. FPGA GPIO

TBD (FPGA HLF)

5.6.4. Memory Interfaces

5.6.4.1. eMMC

IP-50GP board must support a removable Ceragon eMMC card.

5.6.4.2. Secure BOOT

HW to support possibility to burn E-fuse - connect VHV pin to 1.8V in AC5X

5.6.4.3. TPM

Trusted Platform Module (TPM) provides a trusted zone for protected SW update and user data encryption 

5.6.4.4. AC5X Boot Memory

AC5X BOOT is in the eMMC, does not require a serial flash

5.6.4.5. Inventory Serial Flash

Concept as in IP-50EX / IP-100E

Support 8Mb serial Flash, using SPI interface AC5X SPI for quick production access, instead of EEPROM, minimum data writes 10,000 cycles, 32Kbyte blocks

5.6.4.6. DDR

5.6.4.6.1. AC5X DDR

The DDR4 size required for L2 configuration supporting CeraOS + RAON is 4GB. The DDR size to support L3 Configuration is 8GB.

The design shall support maximum required DDR size as an assembly HW ready option. L2 system shall be assembled with 4GB DDRs

For the L2 configuration there shall be 2 chips of DDR4 of 2GB (DDR4 8bit 2400MT/s) with inline ECC shall be connected to the Marvell switch, AC5X.
using inline ECC will reduce BW and capacity by ~ 12%, not need for external DDR for ECC

Internal DDR for parity will be supported

5.6.4.6.2. FPGA DDR

64 bit @2400 MHz interface directly to the FPGA. (2133MHz rate shall be used due to FPGA speed grade)

Four DDR4 16 bit data each

At least 8Mbyte per queue, 8 queues per TM (up-to 4 TMs).

8 x 8 x 4 = 256M, the total size of the DDR4 to be connected to the FPGA shall be at least 256MB

6. Clocks and Synchronization

6.1. System Clock Tree

The system clock tree shall be depicted in

image-20240415-132054.png

6.2. Synchronization

System synchronization diagram is depicted in .

image-20240415-124114.png

6.2.1. CLU Connections 

CLU support three PLLs:

image-20240415-150837.png

Inputs of recovery clock as described in section Recovery clock model

The outputs of reg_clk_25M are an alternative option to the connection of ref_clk that is shown in .

6.3. Marvel AC5X clock sources

The AC5X have two outputs of recovered clocks from SERDES ports, the frequency of ports is dependent on the SerDes configured rate, see , the applicable interface rates are SGMII (1G), 10G and 25G:

image-20240417-064748.png

1g → 1

10G → 8

25G → 16

6.4. FPGA clock sources

The FPGA have 3 types of rates as sync source in the system -

  1. 2.5G - BBI → 25MHz

  2. 1G → 125MHz / 4 = 31.25Mhz

  3. 10G → 257.8125MHz / 8 = 32.2265625MHz

The 1G and 10G rates shall be similar to the recovery frequency of the AC5X.

The FPGA shall have an input of recovery clock from the Marvell and 4 outputs to the CLU.

6.5. Recovery clock model

Planning will be carried out to support 2 ways of changing the synchronization source in the system.

The first output of the recovered clock shall be connected directly to the CLU of the system.

The second output of the recovered clock shall be connected to the FPGA.

image-20240415-105815.png

6.6. SyncE and PTP support

The system shall support synchronization standards as listed in Synchronizations standards paragraph above.

6.6.1. IEEE-1588 support

For information only

IEEE-1588 BC and 1-step TC  is supported by AC5X switch

The FPGA shall support 1-step TC

7. LEDs

7.1. Power Supply LEDs

7.2. Management and protection Ports LEDs

Each of the management and protection ports will have one link/activity LED, these LEDs will be driven by management switch.

7.3. User Ports  LEDs

7.3.1. AC5X LED controller configuration

As in IP-50EX and IP-100E

Single LED will be used as LED indication for link and activity, The controlled configuration as follows:

7.3.1.1. Link and Activity in Single LED Class

By using a single indication, the LED stream length is optimized according to the number of ports.
The most common use case is when the LED indicates both Link and Activity.
This section explains how to configure LED indication class 2 to reflect both Link and Activity in a
single indication. A single indication must reflect the status as follows:

This is achieved by:

  1. Selecting the Activity indication

  2. Causing the Activity indication to blink

  3. Inverting the Activity indication

  4. Disabling the Activity indication on link down

When there is no activity, the Activity indication is 0. Blinking has no affect. Inverting causes it to be 1. If the link is up, it stays 1. If the link is down, it is cleared to 0.
When there is an activity, the Activity indication is 1. Blinking causes it to toggle. Inverting has no meaning (due to toggling). Since the link is up, it is not disabled, thus keeps it in toggling state
(blinking).


All ports connected to the AC5X, shell be mapped to LED class 2 

See section 60 of 98DX35xx functional data sheet for further details 

7.4. FPGA Ports LEDs

In close proximity to each of the SFP+ connectors there is required to be a green LED.

Its functionality is determined by the configuration of the port.

7.4.1. Port Configured as BBI

7.4.2. Port Configured as Ethernet 

7.5. On-board indication LEDs

7.5.1. FPGA Status LED

FPGA status LED - conf_done LED.

The LED shall be green once the FPGA finished loading and the conf_done signal was raised.

7.5.2. Status LED

Status LED is an on-board bi-color LED, functionality according to the table below:

LED

Indication

Remarks

Off

No power

When the unit has power, the LED shouldn't be off

Red

Unit in initialization stage

LED is lit red by HW and should remain in this color until the unit completes Init / Booting

Red blinking

Unit has major/critical alarm

The LED will blink RED when unit have critical/major alarms.

Green

Active Unit is up and all enabled

No critical/major alarms

Green blinking

Standby Unit is up and all enabled

No critical/major alarms

The status LED light and blink controlled by Software

8. DDM

DDM shall be supported as in former platforms
DDM shall be used for establishing the SFP information, temp , voltage … .
All SFP configuration and status shall be managed via FPGA I2C and GPIOs. Support Tx fault, Rx LOS and Tx mute.

9. Power - update

10. Reset

Timing requirements for POR input to power outputs is 100mSec.

Timing requirements for POR input to reference clks and power inputs is at least 10mSec, 50mSec is required active with clocks active.

Reset Name

Reset source 

Components

POR

Marvell POR GPIO

POR restart to all board and power supply

System Watch Dog

TLP5010 (system watch dog device)

Software through FPGA to set “done” to the system watchdog

eMMC RST

Marvell GPIO

eMMC

FPGA RST

Marvell GPIO

FPGA, state machine reset 

AC5X DDR RST

AC5X DDR I/F

AC5X DDR

FPGA DDR RST

FPGA DDR I/F

FPGA DDR

FPGA RBF reset

Marvell GPIO

FPGA

11. System BOOT

The system BOOT shall be done in parallel as possible to save system bring-up time, bring up time affects production time and cost.

The FPGA shall load from AC5X SPI

The AC5x shall BOOT from the eMMC.

11.1. FPGA configuration

MSEL configuration pins shall be set to PS at standard speed.

image-20240416-143814.png

The FPGA configuration scheme shall be PS Device (Passive Serial)

image-20240416-143532.png

The configuration and Initialization stages shall use the DCLK from the SPI, see

12. Environment Requirements

Temperature sensor in power supply, the CPU shall monitor the temperature. Temperature sensor shall give temperature reading for remote access.
When temperature rises above a given threshold, system's power shall be shutdown.

13. Debug capabilities

13.1. JTAG

The following JTAG targets shall be supported by one JTAG connector:

In case debug of one of the above targets is required for debug than a jumper assembly change will be required to direct the JTAG connector to that target.

13.2. UART

Single on board UART connectors for AC5X CPU direct terminal access UART.

13.3. USB

The requirements shall be defined at L3 phase of the platform. Currently the USB is assembly ready only.

14. Testability

14.1. Ethernet

15. Production Support

BIST requirements shall be define in a dedicated HLF





g