Insights into 5G NSA Deployment and Performance in Rome
5G Coverage and Performance
From 4G to 5G Transition
4G
5G
EPC
5G-CN
Today – 4G Access
Device attaches to LTE/4G
radio and Evolved Packet
Core (EPC)
Early 5G – Non-Standalone
Device attaches to 5G-NR,
which routes either via 4G
Base Station to EPC, or
direct to EPC
4G acts as control plane
5G Standalone
Device attaches to 5G-NR
and 5G Core Network.
From 4G to 5G Transition
5G NSA field trials and testing
⮚
5G Non-Standalone (NSA) is being largely adopted for the rollout
of 5G networks
⮚
It is key to investigate how 5G NSA performs in the wild
⮚
Large-scale measurement campaign on 5G NSA deployments for
four Mobile Network Operators (MNOs) in Rome, Italy
⮚
Empirical analysis of the 5G
coverage
, deployment, and
application performance
Measurement Setup and Campaign
⮚
Four MNOs (Op1-Op4) offering 4G and
5G NSA connectivity in Rome, Italy
⮚
Measurements period: 2 months (early
April 2023 - late May 2023)
⮚
Campaign organized in “sub-
campaigns”: different
days
, different
times
of the day and different
locations
⮚
Four mobility scenarios:
⮚
Indoor Static (IS): measurements indoor
⮚
Indoor Walking (IW) measurements while
walking indoor
⮚
Outdoor Walking (OW): measurements
while walking outdoor
⮚
Outdoor Driving (OD): measurements while
driving a car
(a)
RF antenna
(b)
GPS antenna
(c)
R&S TSMA6 scanner
(d)
5G capable UE
For
network analysis
,
troubleshooting
,
visualization
, and
data exporting
,
we used the
R&S ROMES
software
Passive Measurements
⮚
TSMA6 detects and decodes downlink control information
⮚
Measurements 4G and 5G at each location
⮚
Dataset features:
⮚
Spatial (Latitude and Longitude) and temporal (Time)
information
⮚
Carrier frequency identifiers
⮚
Signal strength/quality indicators
Carrier frequency identifiers
https://www.spectrummonitoring.com/frequencies.php?market=I
▪
R
S
R
P
:
I
n
d
i
c
a
t
e
s
t
h
e
a
v
e
r
a
g
e
p
o
w
e
r
o
f
t
h
e
r
e
c
e
i
v
e
d
r
e
f
e
r
e
n
c
e
s
i
g
n
a
l
s
p
r
e
a
d
o
v
e
r
t
h
e
f
u
l
l
b
a
n
d
w
i
d
t
h
.
I
t
i
s
a
f
f
e
c
t
e
d
b
y
d
i
s
t
a
n
c
e
,
n
e
a
r
b
y
b
u
i
l
d
i
n
g
s
,
w
a
l
l
s
,
w
e
a
t
h
e
r
c
o
n
d
i
t
i
o
n
s
,
e
t
c
.
−
Range:
-144dBm (bad) to -44dBm (good)
▪
R
S
R
Q
:
I
n
d
i
c
a
t
e
s
t
h
e
q
u
a
l
i
t
y
o
f
t
h
e
r
e
c
e
i
v
e
d
r
e
f
e
r
e
n
c
e
s
i
g
n
a
l
.
I
t
i
s
d
e
f
i
n
e
d
a
s
R
S
R
Q
=
N
x
R
S
R
P
/
R
S
S
I
,
w
h
e
r
e
N
i
s
t
h
e
n
u
m
b
e
r
o
f
P
h
y
s
i
c
a
l
R
e
s
o
u
r
c
e
B
l
o
c
k
s
.
−
Range:
-19.5dB (bad) to -3dB (good)
▪
S
I
N
R
:
I
n
d
i
c
a
t
e
s
t
h
e
s
i
g
n
a
l
-
t
o
-
n
o
i
s
e
r
a
t
i
o
o
f
t
h
e
r
e
c
e
i
v
e
d
s
i
g
n
a
l
.
I
n
o
t
h
e
r
w
o
r
d
s
,
i
t
m
e
a
s
u
r
e
s
t
h
e
r
a
t
i
o
o
f
t
h
e
d
e
s
i
r
e
d
s
i
g
n
a
l
p
o
w
e
r
t
o
t
h
e
s
u
m
o
f
t
h
e
p
o
w
e
r
o
f
t
h
e
i
n
t
e
r
f
e
r
i
n
g
s
i
g
n
a
l
s
a
n
d
c
a
n
b
e
u
s
e
d
t
o
m
e
a
s
u
r
e
t
h
e
q
u
a
l
i
t
y
o
f
t
h
e
c
o
n
n
e
c
t
i
o
n
−
Range:
-20dB (bad) to 20dB (good)
❏
R
e
s
p
e
c
t
i
v
e
s
i
g
n
a
l
i
n
d
i
c
a
t
o
r
r
a
n
g
e
s
f
o
r
5
G
s
l
i
g
h
t
l
y
d
i
f
f
e
r
.
Signal Indicators
5G Deployment
Spatial representation of the radio network deployment for 𝑂𝑝1. The blue and red
markers indicate the estimated positions of 4G and 5G base stations, respectively. The
black line in the background indicates the routes during the entire measurement
campaign.
4G Coverage
Distribution of 4G RSRP [dBm] across carrier frequencies (in a letter-value plot format
and order by highest to lowest frequency)
5G Coverage
Distribution of 5G SS-RSRP [dBm] (in an ecdf format), for 𝑂𝑝1 and 𝑂𝑝2 and grouped by
scenario.
Significance Tests
▪
K
r
u
s
k
a
l
-
W
a
l
l
i
s
(
n
o
n
-
p
a
r
a
m
e
t
r
i
c
)
:
D
e
t
e
r
m
i
n
e
s
t
a
t
i
s
t
i
c
a
l
s
i
g
n
i
f
i
c
a
n
c
e
b
e
t
w
e
e
n
t
h
e
m
e
a
n
s
o
f
m
o
r
e
t
h
a
n
t
w
o
i
n
d
e
p
e
n
d
e
n
t
g
r
o
u
p
s
A
s
s
u
m
p
t
i
o
n
s
:
•
No assumptions
−
N
u
l
l
H
y
p
o
t
h
e
s
i
s
:
T
h
e
m
e
a
n
s
a
c
r
o
s
s
a
l
l
g
r
o
u
p
s
a
r
e
e
q
u
a
l
−
A
l
t
e
r
n
a
t
i
v
e
H
y
p
o
t
h
e
s
i
s
:
A
t
l
e
a
s
t
o
n
e
g
r
o
u
p
m
e
a
n
i
s
d
i
f
f
e
r
e
n
t
f
r
o
m
t
h
e
r
e
s
t
•
if p-value is less than a significance level (e.g., 0.05), we can reject the null
hypothesis
▪
Dunn’s Test (non-parametric): If Kruskal-Wallis shows statistical significance, conduct
Dunn’s test to determine which groups are different (test each combination of groups)
Operator and Technology comparison
Operator
comparison
Technology
comparison
Operators have
significant differences
especially for Indoor
scenarios.
Different technologies
have significant
differences for both
operators.
Behind the Doors:
4G and 5G Datasets
Data Sources
▪
Coverage data:
−
GEach batch of samples sharing the same timestamp/gps
coordinates represent a single measurement point (in space).
−
Each sample holds information for the UE - cell connection (e.g.,
GPS UE location, RSRP, RSRQ, SINR, cell identifiers, etc.)
▪
Cell data:
(will not be provided for this module)
−
A separate set of data with additional information for each cell
(e.g., GPS cell location, power, cell identifiers, etc.)
Data Structure
▪
13 different locations in Rome, Italy
•
Several iterations for each location (each iteration uses a
different active measurements setup – not relevant for this
module)
▪
2 .csv files (coverage + cell) under each sub-folder +
additional files exported by ROMES
Post-Processing challenges
▪
Cleaning
:
Remove metadata and measurement
configuration details
▪
Missing Data
:
How to handle?
−
e.g., MCN missing - Action: insert it by using other
features (such as Frequency)
−
e.g., GPS missing - Action: if indoor campaign, insert it
manually
▪
Mutate Data
:
Add features useful for the analysis
−
e.g., location, mobility scenario, folder structure (for
future segmentation), etc.
▪
Invalid Data
:
Requires domain knowledge
−
Look for outliers and extreme values
•
Determine either extreme (of interest) behavior or
measurement error (next page)
Visualization
▪
Data illustration with map visualization tools
−
Identify the walking/driving traces
•
check for faulty GPS
−
Pinpoint areas with low coverage at glance
−
Visualize the
cell spatial distribution
−
Can be generalized or expanded with new data
Visualization of GPS location
Visualization of Coverage (SS-RSRP) Op1
Visualization of Coverage (RSRP) - Band
20
Visualization of Coverage (RSRP) - Band 7
Assignment
▪
Data
will be
provided in an R file format
(.RDS)
−
2 files (4G/5G data)
•
185MB
(4G),
315MB
(5G)
−
You have to read both files in R (or python using the
pyreadr
wrapper)
•
You can later export to .csv (if desired)
▪
Each sample represents a passive scan
−
i.e.,
given a location, a unique
cell Identity
and
a non-unique PCI
are
reported alongside signal related information (
next slide for a
complete set of features
):
•
cell Identity
indicators are not always present in 5G (approx ⅔ of
the times missing) - due to on-going deployment of NSA
Dataset Statistics
▪
4G: 13.665.682 samples
▪
5G: 27.213.859 samples
Locations
Scenarios
Dataset Statistics
Dataset Statistics
Operators
Features (4G
)
Example
4G
5G
What to evaluate
▪
Coverage performance
▪
Given a selected number of different (sub-)campaigns, compare the
coverage performance
of
different operators
(further dissect per frequency
band)
and different technologies (
4G, 5G
).
▪
Given two or more sub-campaigns at the same location (e.g., repetitions of
indoor/outdoor campaigns), show how the coverage changes over time.
▪
Given a single operator, compare the coverage performance across
different locations or scenarios. HINT: Particularly interesting for OD
campaigns
▪
Other aspects to evaluate:
•
Does speed affect coverage performance
•
What
beamforming strategies are followed
by each
operator
•
Feel free to evaluate other aspects of the data
▪
Analysis of variance. How significantly different are the different data
groups?
Contact Points
▪
ozgua@ifi.uio.no
▪
konstako@ifi.uio.no
Explore the transition from 4G to 5G, specifically focusing on the deployment and performance of 5G Non-Standalone (NSA) networks in Rome, Italy. Detailed analysis includes coverage, application performance, and empirical data from large-scale measurement campaigns conducted by four Mobile Network Operators (MNOs). The setup involved various mobility scenarios and passive measurements using advanced tools for network analysis and troubleshooting.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
5G Coverage and Performance IN5060
From 4G to 5G Transition From 4G to 5G Transition Today 4G Access Device attaches to LTE/4G radio and Evolved Packet Core (EPC) EPC 4G Early 5G Non-Standalone Device attaches to 5G-NR, which routes either via 4G Base Station to EPC, or direct to EPC 4G acts as control plane 5G Standalone Device attaches to 5G-NR and 5G Core Network. 5G-CN 5G IN5060
5G NSA field trials and testing 5G Non-Standalone (NSA) is being largely adopted for the rollout of 5G networks It is key to investigate how 5G NSA performs in the wild Large-scale measurement campaign on 5G NSA deployments for four Mobile Network Operators (MNOs) in Rome, Italy Empirical analysis of the 5G coverage, deployment, and application performance IN5060
Measurement Setup and Campaign Four MNOs (Op1-Op4) offering 4G and 5G NSA connectivity in Rome, Italy Measurements period: 2 months (early April 2023 - late May 2023) Campaign organized in sub- campaigns : different days, different times of the day and different locations Four mobility scenarios: Indoor Static (IS): measurements indoor Indoor Walking (IW) measurements while walking indoor Outdoor Walking (OW): measurements while walking outdoor Outdoor Driving (OD): measurements while driving a car (a) RF antenna (b) GPS antenna (c) R&S TSMA6 scanner (d) 5G capable UE For network analysis, troubleshooting, visualization, and data exporting, we used the R&S ROMES software IN5060
Passive Measurements TSMA6 detects and decodes downlink control information Measurements 4G and 5G at each location Dataset features: Spatial (Latitude and Longitude) and temporal (Time) information Carrier frequency identifiers Signal strength/quality indicators IN5060
Carrier frequency identifiers https://www.spectrummonitoring.com/frequencies.php?market=I IN5060
Signal Indicators RSRP: Indicates the average power of the received reference signal spread over the full bandwidth. It is affected by distance, nearby buildings, walls, weather conditions, etc. Range: -144dBm (bad) to -44dBm (good) RSRQ: Indicates the quality of the received reference signal. It is defined as RSRQ = N x RSRP / RSSI, where N is the number of Physical Resource Blocks. Range: -19.5dB (bad) to -3dB (good) SINR: Indicates the signal-to-noise ratio of the received signal. In other words, it measures the ratio of the desired signal power to the sum of the power of the interfering signals and can be used to measure the quality of the connection Range: -20dB (bad) to 20dB (good) Respective signal indicator ranges for 5G slightly differ. IN5060
5G Deployment Spatial representation of the radio network deployment for ??1. The blue and red markers indicate the estimated positions of 4G and 5G base stations, respectively. The black line in the background indicates the routes during the entire measurement campaign. IN5060
4G Coverage Distribution of 4G RSRP [dBm] across carrier frequencies (in a letter-value plot format and order by highest to lowest frequency) IN5060
5G Coverage Distribution of 5G SS-RSRP [dBm] (in an ecdf format), for ??1 and ??2 and grouped by scenario. IN5060
Significance Tests Kruskal-Wallis (non-parametric): Determine statistical significance between the means of more than two independent groups Assumptions: No assumptions Null Hypothesis: The means across all groups are equal Alternative Hypothesis: At least one group mean is different from the rest if p-value is less than a significance level (e.g., 0.05), we can reject the null hypothesis Dunn s Test (non-parametric): If Kruskal-Wallis shows statistical significance, conduct Dunn s test to determine which groups are different (test each combination of groups) IN5060
Operator and Technology comparison Technology IS IW OW OD 0.2906 0.0274 * 0.2480 0.3581 LTE 0.1594 0.00033 *** 0.1152 0.1439 5G Operators have significant differences especially for Indoor scenarios. Operator comparison Operator IS IW OW OD 0.0021 ** 2.3e-09 *** 0.0117 * 1e-05 *** 1 0.0133 * 1e-06 *** 0.00078 *** 1e-05 *** 2 Different technologies have significant differences for both operators. Technology comparison IN5060
Behind the Doors: 4G and 5G Datasets IN5060
Data Sources Coverage data: GEach batch of samples sharing the same timestamp/gps coordinates represent a single measurement point (in space). Each sample holds information for the UE - cell connection (e.g., GPS UE location, RSRP, RSRQ, SINR, cell identifiers, etc.) Cell data: (will not be provided for this module) A separate set of data with additional information for each cell (e.g., GPS cell location, power, cell identifiers, etc.) IN5060
Data Structure 13 different locations in Rome, Italy Several iterations for each location (each iteration uses a different active measurements setup not relevant for this module) 2 .csv files (coverage + cell) under each sub-folder + additional files exported by ROMES IN5060
Post-Processing challenges Cleaning: Remove metadata and measurement configuration details Missing Data: How to handle? e.g., MCN missing - Action: insert it by using other features (such as Frequency) e.g., GPS missing - Action: if indoor campaign, insert it manually Mutate Data: Add features useful for the analysis e.g., location, mobility scenario, folder structure (for future segmentation), etc. Invalid Data: Requires domain knowledge Look for outliers and extreme values Determine either extreme (of interest) behavior or measurement error (next page) IN5060
Visualization Data illustration with map visualization tools Identify the walking/driving traces check for faulty GPS Pinpoint areas with low coverage at glance Visualize the cell spatial distribution Can be generalized or expanded with new data IN5060
Visualization of Coverage (RSRP) - Band 20 IN5060
Assignment IN5060
Dataset Statistics Data will be provided in an R file format (.RDS) 2 files (4G/5G data) 185MB (4G), 315MB (5G) You have to read both files in R (or python using the pyreadr wrapper) You can later export to .csv (if desired) Each sample represents a passive scan i.e., given a location, a unique cell Identity and a non-unique PCI are reported alongside signal related information (next slide for a complete set of features): cell Identity indicators are not always present in 5G (approx of the times missing) - due to on-going deployment of NSA IN5060
Dataset Statistics 4G: 13.665.682 samples 5G: 27.213.859 samples Locations 1 2 3 4 6 7 8 9 10 11 12 13 15 IS IS IS OW/OD IS/IW OD IS OD OD IS IS IS IS 4G 1.55% 7.94% 3.26% 7.56% 10.71% 0.99% 0.80% 0.46% 0.78% 0.77% 30.66% 16.13% 18.34% 5G 0.62% 6.17% 4.30% 12.04% 10.90% 0.74% 0.81% 0.53% 0.69% 0.16% 31.30% 15.68% 16.00% Scenarios IS 71.83% 68.94% IW 0.94% 1.18% OW 0.49% 0.41% OD 26.73% 29.44% 4G 5G IN5060
Dataset Statistics Operators Tim 33.66% 5.82% Vodafone 33.39% 44.24% Iliad 14.01% 18.29% Wind 18.92% 31.63% 4G 5G IN5060
Features (4G) Feature Date Time Latitude Longitude Speed EARFCN Frequency PCI MCC MNC Brief Description Measurement Date Measurement Time UE Latitude UE Longitude Moving Speed [Km/h] Band-Carrier Frequency Carrier Frequency [MHz] Physical Cell Identifier Mobile Country Code Mobile Network Code Feature eNodeB.ID (4G-only) Power/SS-Power SINR/SS-SINR RSRP/SS-RSRP RSRQ/SS-RSRQ location workspace scenario SSB-Idx (5G-only) RNC-CellID(H) (5G- only) RNC-CellID(D) (5G- only) Brief Description Unique eNodeB Identifier Power Received on the Entire Bandwidth [dBm] Signal-to-Interference-plus-Noise Ratio [dB] Reference Signal Receive Power [dBm] Reference Signal Received Quality [dB] Location name Workspace name (iteration) Scenario (IS/IW/OW/OD) Beam Indicator Cell indicator CI (4G-only) Unique CellID Identifier Cell indicator IN5060
Example 4G 5G IN5060
What to evaluate Coverage performance Given a selected number of different (sub-)campaigns, compare the coverage performance of different operators (further dissect per frequency band) and different technologies (4G, 5G). Given two or more sub-campaigns at the same location (e.g., repetitions of indoor/outdoor campaigns), show how the coverage changes over time. Given a single operator, compare the coverage performance across different locations or scenarios. HINT: Particularly interesting for OD campaigns Other aspects to evaluate: Does speed affect coverage performance What beamforming strategies are followed by each operator Feel free to evaluate other aspects of the data Analysis of variance. How significantly different are the different data groups? IN5060
Contact Points ozgua@ifi.uio.no konstako@ifi.uio.no IN5060
