Plantronics + Polycom. Now together as Poly Logo

!Global Skype for Business Issue!

Polycom Employee & Community Manager

!Global Skype for Business Issue!

Hello all,

 

we seem to have multiple reports of Skype for Business online sign-in issues.

 

From an affected Customer log we see:

1217114726|curl |1|00|HEADER_IN : report-to: {"group":"network-errors","max_age":86400,"endpoints":[{"url":"https://identity.nelreports.net/api/report?catId=GW+estsfd+dub2"}]}
1217114726|curl |1|00|HEADER_IN : nel: {"report_to":"network-errors","max_age":86400,"success_fraction":0.001,"failure_fraction":1.0}
1217114726|curl |1|00|HEADER_IN : Set-Cookie: fpc=Ag6hGzYAUaZIjGHUbIAugW4; expires=Sat, 16-Jan-2021 11:47:27 GMT; path=/; secure; HttpOnly; SameSite=None
1217114726|curl |1|00|HEADER_IN : Set-Cookie: x-ms-gateway-slice=estsfd; path=/; secure; httponly
1217114726|curl |1|00|HEADER_IN : Set-Cookie: stsservicecookie=estsfd; path=/; secure; httponly
1217114726|curl |1|00|HEADER_IN : Content-Disposition: inline; filename=userrealm.json
1217114726|curl |1|00|HEADER_IN : Date: Thu, 17 Dec 2020 11:47:26 GMT
1217114726|curl |1|00|HEADER_IN : Content-Length: 175

Checking the URL https://identity.nelreports.net/api/report?catId=GW+estsfd+dub2 we get:

 

  • Not able find PartnerConfig for Host: identity.nelreports.net Path: /api/report Query:Microsoft.AspNetCore.Http.Internal.QueryCollection

To verify your issue is the same:

 

Settings > Logging > Global Settings > Global Log Level Limit > DebugSettings > Logging > Global Settings > Global Log Level Limit > Log to Standard Output > EnableSettings > Logging > Global Settings > Global Log Level Limit > Log File Size (Kbytes) > Trio 8300 & VVX after 5.5.0 = 1000Settings > Logging > Global Settings > Global Log Level Limit > Log File Size (Kbytes) > Trio or CCX 10240
Settings > Logging > Module Log Level Limits > CURL > Event 1
Settings > Logging > Module Log Level Limits > SIP > Debug

 

We will update this once we know more

 

Best Regards

 

Steffen Baier

----------------
The title Polycom Employee & Community Manager is a community setting and does not reflect my role. I am just a simple volunteer in the community like everybody else. My official "day" Job is 3rd Level support at Poly but I am unable to provide official support via the community.

----------------

Notice: This community forum is not an official Poly support resource, thus responses from Poly employees, partners, and customers alike are best-effort in attempts to share learned knowledge. If you need immediate and/or official assistance please open a service ticket through your proper support channels.
Please also ensure you always check the VoIP , Video Endpoint , Skype for Business , PSTN or RPM FAQ's
Message 1 of 6
5 REPLIES 5
Occasional Visitor

Re: !Global Skype for Business Issue!

Thanks for the update, this morning everyone started emailing that they can't seem to login to the phones.

 

VVX 401 phones are not registering, authentication failed. 

 

Please update ASAP, or if we need to perform a firmware upgrade.

 

Thank you

Message 2 of 6
Polycom Employee & Community Manager

Re: !Global Skype for Business Issue!

Hello @plateplus 

 

at present, it does not look to be a Poly issue and we are working with our Partner Microsoft on this.

 

Its a global issue affecting certain tenants. The Microsoft Ticket reference is 23363356 in case you want to open your own ticket.

 

Have a look at this post: 

 

Jun 05, 2018 Question: Do Poly Phones support a REST API?

Resolution: Please => here <=

 

The above has some examples on how to use the REST API in case you need phones to reboot so they can sign in again once Microsoft has identified the root cause.

 

Best Regards

 

Steffen Baier

----------------
The title Polycom Employee & Community Manager is a community setting and does not reflect my role. I am just a simple volunteer in the community like everybody else. My official "day" Job is 3rd Level support at Poly but I am unable to provide official support via the community.

----------------

Notice: This community forum is not an official Poly support resource, thus responses from Poly employees, partners, and customers alike are best-effort in attempts to share learned knowledge. If you need immediate and/or official assistance please open a service ticket through your proper support channels.
Please also ensure you always check the VoIP , Video Endpoint , Skype for Business , PSTN or RPM FAQ's
Message 3 of 6
Polycom Employee & Community Manager

Re: !Global Skype for Business Issue!

Hello all,

 

our Partner Microsoft is still investigating the root cause on this. It is now stated in the Office 365 Admin Center

image.pngimage.png

 

As a workaround, once the fault is fixed, we assume phones have to be restarted.

 

Workaround 1:

 

Prerequisite the phones must have the REST API already enabled. This covers users on Poly RealPresence Resource Manager or PDMS-E

 

Get a list of all phones IP addresses from the DHCP server as an example and store them in a text file called ip.txt

 

Copy the below and ensure to change the password 789 to what the actual admin password of the phone is.

 

@echo off
echo Reading ip.txt
for /F "tokens=1" %i in (ip.txt) do (
curl -i -X POST -H "Content-Type: application/json" -d "" -k https://Polycom:789@%i/api/v1/mgmt/safeRestart
timeout /t 5 /NOBREAK
)

 

Store the above as reboot.bat and store it in the same folder as ip.txt. Open a MS-Dos command line and change to the directory where both files have been stored. run reboot.bat

(It only works in newer versions of Windows 10 that have CURL)

 

Workaround 2:

 

Below is a PERL script which can be used in a local network. The default username is Polycom and the Password can be changed from 789:

 

 

#!/usr/bin/perl -w
# RebootScript.pl - Reboot range of Polycom phones using the Web Interface
# Initial Script (???) - bm
# Updated (???) - sb
# Update (28 April 2017) - mcab
# 	- Set the admin username and password at the top of the file. (Script will generate the cookie automatically now)
# 	- Removed dependency on CGI.pm
# Update (28 April 2017 #2) - mcab
# 	- add IO::Socket::SSL
# 	- ignore validating TLS certs from phone


use strict;
use LWP::UserAgent;
use IO::Socket;
use Socket; 
use IO::Socket::SSL;
use MIME::Base64;

my $username = "Polycom";
my $password = "789";

#####################################
# Netmask 255.255.255.0	= 0xFFFFFF00#
# Netmask 255.255.0.0	= 0xFFFF0000#
# Netmask 255.0.0.0		= 0xFF000000#
# Netmask 0.0.0.0		= 0x00000000#
#####################################

my $netmask = 0xFFFFFF00; # 255.255.255.0

#########################################
# Define IP Address Start Range	$Start	#
# Define IP Address End Range $finish	#
#########################################

my $start_address = unpack 'N', inet_aton( '10.252.149.57' );
my $finish_address = unpack 'N', inet_aton( '10.252.149.58' );


for ( my $address = $start_address; $address <= $finish_address; ++$address ) {
next if ( $address & $netmask ) == $address or ( $address & ~$netmask ) == ~$netmask;


my $ua = LWP::UserAgent->new( );
$ua->ssl_opts( verify_hostname => 0, SSL_verify_mode => SSL_VERIFY_NONE );
my $destination = inet_ntoa( pack 'N', $address);

#############################
# Define Reboot or Restart  #
# Simply remove # and add # #
#############################

my $restartUrl = "https://$destination/form-submit/Reboot";
  
# Create request object and add the authentication header and the expected cookie
  my $req =  HTTP::Request->new(POST => $restartUrl);
  $req->header('Cookie' => 'Authorization=Basic ' . encode_base64("$username:$password"));
  $req->authorization_basic( $username, $password );
 
 
# Send the reset request to the user agent
  my $res = $ua->request( $req );
  print $res->status_line, "\n";
  sleep (5)
}

 

 

Store this on a PC and run it (needs Perl installed)

 

This simple and basic Perl script can enable the Reboot of a range of Phones

How does it work?

 

2 Actions are required within the Script itself:

- Define the Password ($Password)
my $password = "789";
- Define the IP Address Range (aka start / stop)

Example: 10.252.149.57 till 10.252.149.58
my $start_address = unpack 'N', inet_aton( '10.252.149.57' );
my $finish_address = unpack 'N', inet_aton( '10.252.149.58' );

 

How do I run it?

- Download and install ActivePerl for Windows PC'script
- Download RebootScript.pl.txt and store as RebootScript.pl into the C:\Perl64\bin directory
- Open a CMD window and browse to C:\Perl64\bin and Edit to suit the above actions
- perl RebootScript.pl and hit enter. It will not send any warnings to anyone and wait 5 Seconds between Phone IP

----------------
The title Polycom Employee & Community Manager is a community setting and does not reflect my role. I am just a simple volunteer in the community like everybody else. My official "day" Job is 3rd Level support at Poly but I am unable to provide official support via the community.

----------------

Notice: This community forum is not an official Poly support resource, thus responses from Poly employees, partners, and customers alike are best-effort in attempts to share learned knowledge. If you need immediate and/or official assistance please open a service ticket through your proper support channels.
Please also ensure you always check the VoIP , Video Endpoint , Skype for Business , PSTN or RPM FAQ's
Message 4 of 6
Polycom Employee & Community Manager

Re: !Global Skype for Business Issue!

Hello all,

 

this seems to be resolved via a change on our Partner Microsofts end. Please work with Microsoft for any follow-up questions.

 

Best regards

 

Steffen Baier

----------------
The title Polycom Employee & Community Manager is a community setting and does not reflect my role. I am just a simple volunteer in the community like everybody else. My official "day" Job is 3rd Level support at Poly but I am unable to provide official support via the community.

----------------

Notice: This community forum is not an official Poly support resource, thus responses from Poly employees, partners, and customers alike are best-effort in attempts to share learned knowledge. If you need immediate and/or official assistance please open a service ticket through your proper support channels.
Please also ensure you always check the VoIP , Video Endpoint , Skype for Business , PSTN or RPM FAQ's
Message 5 of 6
Polycom Employee & Community Manager

Re: !Global Skype for Business Issue!

The official explanation from the O365 Admin Centre (LY230423):

 

Title: Users can't sign into Poly devices

User Impact: Users may have been unable to sign into Poly devices.

More info: This was affecting Skype for Business and Teams-only Azure Active Directory (AAD) federated users logging into Poly devices.

Final status: We've rolled back the recently deployed update and confirmed with affected users that impact was remediated.

Scope of impact: Impact was specific to users signing into Poly devices.

Start time: Thursday, December 17, 2020, 12:59 AM (12:59 AM UTC)

End time: Thursday, December 17, 2020, 6:00 PM (6:00 PM UTC)

Root cause: A recently deployed update implemented telemetry enhancements for AAD that resulted in users being unable to sign into Poly devices.

Next steps:
- We're reviewing our update procedures to better identify similar Poly device authentication issues during our development and testing cycles.

This is the final update for the event.

 

We are working with our Partner Microsoft so any change like this in the future should not affect the end-user.

----------------
The title Polycom Employee & Community Manager is a community setting and does not reflect my role. I am just a simple volunteer in the community like everybody else. My official "day" Job is 3rd Level support at Poly but I am unable to provide official support via the community.

----------------

Notice: This community forum is not an official Poly support resource, thus responses from Poly employees, partners, and customers alike are best-effort in attempts to share learned knowledge. If you need immediate and/or official assistance please open a service ticket through your proper support channels.
Please also ensure you always check the VoIP , Video Endpoint , Skype for Business , PSTN or RPM FAQ's
Message 6 of 6