No authenticationScheme was specified, and there was no DefaultChallengeScheme found with default authentification and custom authorization

Do not use authorization instead of authentication. I should get whole access to service all clients with header.

The working code is:

public class TokenAuthenticationHandler : AuthenticationHandler<TokenAuthenticationOptions> 
{
    public IServiceProvider ServiceProvider { get; set; }

    public TokenAuthenticationHandler (IOptionsMonitor<TokenAuthenticationOptions> options, ILoggerFactory logger, UrlEncoder encoder, ISystemClock clock, IServiceProvider serviceProvider) 
        : base (options, logger, encoder, clock) 
    {
        ServiceProvider = serviceProvider;
    }

    protected override Task<AuthenticateResult> HandleAuthenticateAsync () 
    {
        var headers = Request.Headers;
        var token = "X-Auth-Token".GetHeaderOrCookieValue (Request);

        if (string.IsNullOrEmpty (token)) {
            return Task.FromResult (AuthenticateResult.Fail ("Token is null"));
        }           

        bool isValidToken = false; // check token here

        if (!isValidToken) {
            return Task.FromResult (AuthenticateResult.Fail ($"Balancer not authorize token : for token={token}"));
        }

        var claims = new [] { new Claim ("token", token) };
        var identity = new ClaimsIdentity (claims, nameof (TokenAuthenticationHandler));
        var ticket = new AuthenticationTicket (new ClaimsPrincipal (identity), this.Scheme.Name);
        return Task.FromResult (AuthenticateResult.Success (ticket));
    }
}

Startup.cs:

#region Authentication
services.AddAuthentication (o => {
    o.DefaultScheme = SchemesNamesConst.TokenAuthenticationDefaultScheme;
})
.AddScheme<TokenAuthenticationOptions, TokenAuthenticationHandler> (SchemesNamesConst.TokenAuthenticationDefaultScheme, o => { });
#endregion

And mycontroller.cs:

[Authorize(AuthenticationSchemes = SchemesNamesConst.TokenAuthenticationDefaultScheme)]
public class MainController : BaseController
{ ... }

I can't find TokenAuthenticationOptions now, but it was empty. I found the same class PhoneNumberAuthenticationOptions:

public class PhoneNumberAuthenticationOptions : AuthenticationSchemeOptions
{
    public Regex PhoneMask { get; set; }// = new Regex("7\\d{10}");
}

You should define static class SchemesNamesConst. Something like:

public static class SchemesNamesConst
{
    public const string TokenAuthenticationDefaultScheme = "TokenAuthenticationScheme";
}

---

this worked for me

// using Microsoft.AspNetCore.Authentication.Cookies;
// using Microsoft.AspNetCore.Http;

services.AddAuthentication(CookieAuthenticationDefaults.AuthenticationScheme)
    .AddCookie(CookieAuthenticationDefaults.AuthenticationScheme,
        options =>
        {
            options.LoginPath = new PathString("/auth/login");
            options.AccessDeniedPath = new PathString("/auth/denied");
        });

---

When I used policy before I set the default authentication scheme into it as well. I had modified the DefaultPolicy so it was slightly different. However the same should work for add policy as well.

services.AddAuthorization(options =>
{
    options.AddPolicy(DefaultAuthorizedPolicy, policy =>
    {
        policy.Requirements.Add(new TokenAuthRequirement());
        policy.AuthenticationSchemes = new List<string>()
        {
            CookieAuthenticationDefaults.AuthenticationScheme
        }
    });
});

Do take into consideration that by Default AuthenticationSchemes property uses a read only list. I think it would be better to implement that instead of List as well.

---

Your initial statement in the marked solution isn't entirely true. While your new solution may accomplish your original goal, it is still possible to circumvent the original error while preserving your AuthorizationHandler logic--provided you have basic authentication scheme handlers in place, even if they are functionally skeletons.

Speaking broadly, Authentication Handlers and schemes are meant to establish + validate identity, which makes them required for Authorization Handlers/policies to function--as they run on the supposition that an identity has already been established.

ASP.NET Dev Haok summarizes this best best here: "Authentication today isn't aware of authorization at all, it only cares about producing a ClaimsPrincipal per scheme. Authorization has to be aware of authentication somewhat, so AuthenticationSchemes in the policy is a mechanism for you to associate the policy with schemes used to build the effective claims principal for authorization (or it just uses the default httpContext.User for the request, which does rely on DefaultAuthenticateScheme)." https://github.com/aspnet/Security/issues/1469

In my case, the solution I'm working on provided its own implicit concept of identity, so we had no need for authentication schemes/handlers--just header tokens for authorization. So until our identity concepts changes, our header token authorization handlers that enforce the policies can be tied to 1-to-1 scheme skeletons.

Tags on endpoints:

[Authorize(AuthenticationSchemes = "AuthenticatedUserSchemeName", Policy = "AuthorizedUserPolicyName")]

Startup.cs:

        services.AddAuthentication(options =>
        {
            options.DefaultAuthenticateScheme = "AuthenticatedUserSchemeName";
        }).AddScheme<ValidTokenAuthenticationSchemeOptions, ValidTokenAuthenticationHandler>("AuthenticatedUserSchemeName", _ => { });

        services.AddAuthorization(options =>
        {
            options.AddPolicy("AuthorizedUserPolicyName", policy =>
            {
                //policy.RequireClaim(ClaimTypes.Sid,"authToken");
                policy.AddAuthenticationSchemes("AuthenticatedUserSchemeName");
                policy.AddRequirements(new ValidTokenAuthorizationRequirement());
            });
            services.AddSingleton<IAuthorizationHandler, ValidTokenAuthorizationHandler>();

Both the empty authentication handler and authorization handler are called (similar in setup to OP's respective posts) but the authorization handler still enforces our authorization policies.

---

Many answer above are correct but same time convoluted with other aspects of authN/authZ. What actually resolves the exception in question is this line:

services.AddScheme<YourAuthenticationOptions, YourAuthenticationHandler>(YourAuthenticationSchemeName, options =>
    {
        options.YourProperty = yourValue;
    })

---