1.First of all, name of the server had to be changed by adding DNS suffix. For example, if you want to connect to the server by srv.example.com address, your server name should be "srv" and DNS suffix "example.com". It can be done in computer properties.
2.Then setup licensing in "RD Licensing Manager"
3.Now issue certificate to domain name srv.example.com (i.e. in Let's encrypt)
4.Convert let's encrypt cert files into windows one via: openssl pkcs12 -export -out certificate.pfx -inkey privkey.pem -in cert.pem -certfile chain.pem (Linux command) if you issued certificate with help of acme.sh, you command should look like: openssl pkcs12 -export -out certificate.pfx -inkey yourdomain.com.key -in yourdomain.com.cer -certfile fullchain.cer
5.Install converted certificate to personal store at the computer level. Not at user level
6.Then with this command display the thumbprint of the certificate, copy it to a text file or something similar: Get-ChildItem "Cert:\LocalMachine\My"
7.This is a variable to set the WMI path to the RD Session Host RDP listener (Where the certificate needs to be changed): $PATH = (Get-WmiObject -class "Win32_TSGeneralSetting" -Namespace root\cimv2\terminalservices)
8.Finally this is the command to change the active certificate on the RDP listener: Set-WmiInstance -Path $PATH -argument @{SSLCertificateSHA1Hash="thumbprint"} The "thumbprint" above, is the value you noted earlier, just insert it between the ".
9.Well done! Now you have RDP server with custom SSL cert without installing RD Gateway Credits