Skip to content

Commit

Permalink
Add t3c flag for local ATS version for config gen (#7032)
Browse files Browse the repository at this point in the history
  • Loading branch information
@rob05c
rob05c authored Aug 26, 2022
1 parent ad27aae commit 3b21783
Show file tree
Hide file tree
Showing 18 changed files with 579 additions and 180 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
## [unreleased]
### Added
- [#6033](https://github.com/apache/trafficcontrol/issues/6033) [Traffic Ops, Traffic Portal] Added ability to assign multiple server capabilities to a server.
- [#7032](https://github.com/apache/trafficcontrol/issues/7032) Add t3c-apply flag to use local ATS version for config generation rather than Server package Parameter, to allow managing the ATS OS package via external tools. See 'man t3c-apply' and 'man t3c-generate' for details.
- [Traffic Monitor] Added logging for `ipv4Availability` and `ipv6Availability` in TM.

### Fixed
Expand Down
202 changes: 110 additions & 92 deletions cache-config/t3c-apply/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,189 +54,207 @@ Typical usage is to install t3c on the cache machine, and then run it periodical

-2, -\-default-client-enable-h2

Whether to enable HTTP/2 on Delivery Services by default, if
they have no explicit Parameter. This is irrelevant if ATS
records.config is not serving H2. If omitted, H2 is
disabled.
Whether to enable HTTP/2 on Delivery Services by default, if
they have no explicit Parameter. This is irrelevant if ATS
records.config is not serving H2. If omitted, H2 is
disabled.

-a, -\-service-action=value

action to perform on Traffic Server and other system
services. Only reloads if necessary, but always restarts.
Default is 'reload'
The action to perform on Traffic Server and other system
services. Only reloads if necessary, but always restarts.
Default is 'reload'

-A, -\-update-ipallow

Whether ipallow file will be updated if necessary. This
exists because ATS had a bug where reloading after changing
ipallow would block everything. Default is false.
Whether ipallow file will be updated if necessary. This
exists because ATS had a bug where reloading after changing
ipallow would block everything. Default is false.

-b, -\-dns-local-bind

[true | false] whether to use the server's Service Addresses
to set the ATS DNS local bind address
[true | false] whether to use the server's Service Addresses
to set the ATS DNS local bind address.

-c, -\-disable-parent-config-comments

Whether to disable verbose parent.config comments. Default
false.
Whether to disable verbose parent.config comments. Default
false.

-C, -\-skip-os-check

[false | true] skip os check, default is false
[false | true] skip os check, default is false

-d, -\-no-unset-update-flag

Whether to not unset the update flag in Traffic Ops after
applying files. This option makes it possible to generate
test or debug configuration from a production Traffic Ops
without un-setting queue or reval flags. Default is false.
Whether to not unset the update flag in Traffic Ops after
applying files. This option makes it possible to generate
test or debug configuration from a production Traffic Ops
without un-setting queue or reval flags. Default is false.

-e, -\-omit-via-string-release

Whether to set the records.config via header to the ATS
release from the RPM. Default true.
Whether to set the records.config via header to the ATS
release from the RPM. Default true.

-E, -\-version

Print version information and exit.
Print version information and exit.

-f, -\-files=value [all | reval]

Which files to generate. If reval, the Traffic
Ops server reval_pending flag is used instead of the
upd_pending flag. Default is 'all'
Which files to generate. If reval, the Traffic
Ops server reval_pending flag is used instead of the
upd_pending flag. Default is 'all'.

-F, -\-ignore-update-flag

Whether to ignore the upd_pending or reval_pending flag in
Traffic Ops, and always generate and apply files. If true,
the flag is still unset in Traffic Ops after files are
applied. Default is false.
Whether to ignore the upd_pending or reval_pending flag in
Traffic Ops, and always generate and apply files. If true,
the flag is still unset in Traffic Ops after files are
applied. Default is false.

-g, -\-git=value
Create and use a git repo in the config directory. Options
are yes, no, and auto. If yes, create and use. If auto, use
if it exist. Default is auto. [auto]

Create and use a git repo in the config directory. Options
are yes, no, and auto. If yes, create and use. If auto, use
if it exist. Default is auto. [auto]

-H, -\-cache-host-name=value

Host name of the cache to generate config for. Must be the
server host name in Traffic Ops, not a URL, and not the FQDN
Host name of the cache to generate config for. Must be the
server host name in Traffic Ops, not a URL, and not the FQDN

-h, -\-help

Print usage information and exit
Print usage information and exit

-i, -\-no-outgoing-ip

Whether to not set the records.config outgoing IP to the
server's addresses in Traffic Ops. Default is false.
Whether to not set the records.config outgoing IP to the
server's addresses in Traffic Ops. Default is false.

-I, -\-traffic-ops-insecure

[true | false] ignore certificate errors from Traffic Ops
[true | false] ignore certificate errors from Traffic Ops

-k, -\-install-packages

Whether to install necessary packages. Default is false.
Whether to install necessary packages. Default is false.

-\-local-ats-version

[true | false] whether to use the local installed ATS version
for config generation. If false, attempt to use the Server
Package Parameter and fall back to ATS 5. If true and the
local ATS version cannot be found, an error will be logged
and the version set to ATS 5. Default is false.

-M, -\-maxmind-location=value

URL of a maxmind gzipped database file, to be installed into
the trafficserver etc directory.
URL of a maxmind gzipped database file, to be installed into
the trafficserver etc directory.

-m, -\-run-mode=value

[badass | report | revalidate | syncds] run mode. Optional, convenience flag which sets other flags for common usage scenarios.
syncds keeps the defaults:
--report-only=false
--files=all
--install-packages=false
--service-action=reload
--ignore-update-flag=false
--update-ipallow=false
--no-unset-update-flag=false
revalidate sets --files=reval
--wait-for-parents=true
badass sets --install-packages=true
--service-action=restart
--ignore-update-flag=true
--update-ipallow=true
report sets --report-only=true

Note the 'syncds' settings are all the flag defaults. Hence, if no mode is set, the default is effectively 'syncds'.

If any of the related flags are also set, they override the mode's default behavior.

-n, -\-no-cache
[badass | report | revalidate | syncds] run mode. Optional,
convenience flag which sets other flags for common usage
scenarios.

syncds keeps the defaults:
--report-only=false
--files=all
--install-packages=false
--service-action=reload
--ignore-update-flag=false
--update-ipallow=false
--no-unset-update-flag=false

revalidate sets --files=reval
--wait-for-parents=true

badass sets --install-packages=true
--service-action=restart
--ignore-update-flag=true
--update-ipallow=true

report sets --report-only=true

Note the 'syncds' settings are all the flag defaults. Hence,
if no mode is set, the default is effectively 'syncds'.

If any of the related flags are also set, they override the
mode's default behavior.

-n, -\-no-cache

Whether to not use a cache and make conditional requests to
Traffic Ops. Default is false: use cache.

-o, -\-report-only

Log information about necessary files and actions, but take
no action. Default is false
Log information about necessary files and actions, but take
no action. Default is false

-p, -\-reverse-proxy-disable

[false | true] bypass the reverse proxy even if one has been
configured default is false
[false | true] bypass the reverse proxy even if one has been
configured default is false

-P, -\-traffic-ops-password=value

Traffic Ops password. Required. May also be set with the
environment variable TO_PASS
Traffic Ops password. Required. May also be set with the
environment variable TO_PASS

-r, -\-num-retries=value

[number] retry connection to Traffic Ops URL [number] times,
default is 3 [3]
[number] retry connection to Traffic Ops URL [number] times,
default is 3 [3]

-R, -\-trafficserver-home=value

Trafficserver Package directory. May also be set with the
environment variable TS_HOME
Trafficserver Package directory. May also be set with the
environment variable TS_HOME

-s, -\-silent

Silent. Errors are not logged, and the 'verbose' flag is
ignored. If a fatal error occurs, the return code will be
non-zero but no text will be output to stderr
Silent. Errors are not logged, and the 'verbose' flag is
ignored. If a fatal error occurs, the return code will be
non-zero but no text will be output to stderr

-t, -\-traffic-ops-timeout-milliseconds=value

Timeout in milli-seconds for Traffic Ops requests, default
is 30000 [30000]
Timeout in milli-seconds for Traffic Ops requests, default
is 30000 [30000]

-u, -\-traffic-ops-url=value

Traffic Ops URL. Must be the full URL, including the scheme.
Required. May also be set with the environment variable
TO_URL
Traffic Ops URL. Must be the full URL, including the scheme.
Required. May also be set with the environment variable
TO_URL

-U, -\-traffic-ops-user=value

Traffic Ops username. Required. May also be set with the
environment variable TO_USER
Traffic Ops username. Required. May also be set with the
environment variable TO_USER

-V, -\-default-client-tls-versions=value

Comma-delimited list of default TLS versions for Delivery
Services with no Parameter, e.g.
--default-tls-versions='1.1,1.2,1.3'. If omitted, all
versions are enabled.
Comma-delimited list of default TLS versions for Delivery
Services with no Parameter, e.g.
--default-tls-versions='1.1,1.2,1.3'. If omitted, all
versions are enabled.

-v, -\-verbose

Log verbosity. Logging is output to stderr. By default,
errors are logged. To log warnings, pass '-v'. To log info,
pass '-vv'. To omit error logging, see '-s' [0]
Log verbosity. Logging is output to stderr. By default,
errors are logged. To log warnings, pass '-v'. To log info,
pass '-vv'. To omit error logging, see '-s' [0]

-W, -\-wait-for-parents

[true | false] do not update if parent_pending = 1 in the
update json. Default is false
[true | false] do not update if parent_pending = 1 in the
update json. Default is false

# MODES

Expand Down
32 changes: 32 additions & 0 deletions cache-config/t3c-apply/config/config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
"net/url"
"os"
"os/exec"
"path/filepath"
"strings"
"time"

Expand Down Expand Up @@ -116,6 +117,7 @@ type Cfg struct {
UpdateIPAllow bool
Version string
GitRevision string
LocalATSVersion string
}

func (cfg Cfg) AppVersion() string { return t3cutil.VersionStr(AppName, cfg.Version, cfg.GitRevision) }
Expand Down Expand Up @@ -278,6 +280,9 @@ func GetCfg(appVersion string, gitRevision string) (Cfg, error) {
const defaultUseStrategies = t3cutil.UseStrategiesFlagFalse
useStrategiesPtr := getopt.EnumLong(useStrategiesFlagName, 0, []string{string(t3cutil.UseStrategiesFlagTrue), string(t3cutil.UseStrategiesFlagCore), string(t3cutil.UseStrategiesFlagCore), ""}, "", "[true | core| false] whether to generate config using strategies.yaml instead of parent.config. If true use the parent_select plugin, if 'core' use ATS core strategies, if false use parent.config.")

const useLocalATSVersionFlagName = "local-ats-version"
useLocalATSVersionPtr := getopt.BoolLong(useLocalATSVersionFlagName, 0, "[true | false] whether to use the local installed ATS version for config generation. If false, attempt to use the Server Package Parameter and fall back to ATS 5. If true and the local ATS version cannot be found, an error will be logged and the version set to ATS 5. Default is false")

const runModeFlagName = "run-mode"
runModePtr := getopt.StringLong(runModeFlagName, 'm', "", `[badass | report | revalidate | syncds] run mode. Optional, convenience flag which sets other flags for common usage scenarios.
syncds keeps the defaults:
Expand Down Expand Up @@ -475,6 +480,15 @@ If any of the related flags are also set, they override the mode's default behav
toInfoLog = append(toInfoLog, fmt.Sprintf("TSHome: %s, TSConfigDir: %s\n", TSHome, tsConfigDir))
}

atsVersionStr := ""
if *useLocalATSVersionPtr {
atsVersionStr, err = GetATSVersionStr(tsHome)
if err != nil {
return Cfg{}, errors.New("getting local ATS version: " + err.Error())
}
}
toInfoLog = append(toInfoLog, fmt.Sprintf("ATSVersionStr: '%s'\n", atsVersionStr))

usageStr := "basic usage: t3c-apply --traffic-ops-url=myurl --traffic-ops-user=myuser --traffic-ops-password=mypass --cache-host-name=my-cache"
if strings.TrimSpace(toURL) == "" {
return Cfg{}, errors.New("Missing required argument --traffic-ops-url or TO_URL environment variable. " + usageStr)
Expand Down Expand Up @@ -539,6 +553,7 @@ If any of the related flags are also set, they override the mode's default behav
NoUnsetUpdateFlag: *noUnsetUpdateFlagPtr,
Version: appVersion,
GitRevision: gitRevision,
LocalATSVersion: atsVersionStr,
}

if err = log.InitCfg(cfg); err != nil {
Expand Down Expand Up @@ -604,6 +619,22 @@ func getOSSvcManagement() SvcManagement {
return _svcManager
}

func GetATSVersionStr(tsHome string) (string, error) {
tsPath := tsHome
tsPath = filepath.Join(tsPath, "bin")
tsPath = filepath.Join(tsPath, "traffic_server")

stdOut, stdErr, code := t3cutil.Do(`sh`, `-c`, `set -o pipefail && `+tsPath+` --version | head -1 | awk '{print $3}'`)
if code != 0 {
return "", fmt.Errorf("traffic_server --version returned code %v stderr '%v' stdout '%v'", code, string(stdErr), string(stdOut))
}
atsVersion := strings.TrimSpace(string(stdOut))
if atsVersion == "" {
return "", fmt.Errorf("traffic_server --version returned nothing, code %v stderr '%v' stdout '%v'", code, string(stdErr), string(stdOut))
}
return atsVersion, nil
}

func printConfig(cfg Cfg) {
// TODO add new flags
log.Debugf("LogLocationDebug: %s\n", cfg.LogLocationDebug)
Expand All @@ -621,6 +652,7 @@ func printConfig(cfg Cfg) {
log.Debugf("TOPass: Pass len: '%d'\n", len(cfg.TOPass))
log.Debugf("TOURL: %s\n", cfg.TOURL)
log.Debugf("TSHome: %s\n", TSHome)
log.Debugf("LocalATSVersion: %s\n", cfg.LocalATSVersion)
log.Debugf("WaitForParents: %v\n", cfg.WaitForParents)
log.Debugf("YumOptions: %s\n", cfg.YumOptions)
log.Debugf("MaxmindLocation: %s\n", cfg.MaxMindLocation)
Expand Down
3 changes: 3 additions & 0 deletions cache-config/t3c-apply/torequest/cmd.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,9 @@ func generate(cfg config.Cfg) ([]t3cutil.ATSConfigFile, error) {
if cfg.Files == t3cutil.ApplyFilesFlagReval {
args = append(args, "--revalidate-only")
}
if cfg.LocalATSVersion != "" {
args = append(args, "--ats-version="+cfg.LocalATSVersion)
}
args = append(args, "--via-string-release="+strconv.FormatBool(!cfg.OmitViaStringRelease))
args = append(args, "--no-outgoing-ip="+strconv.FormatBool(cfg.NoOutgoingIP))
args = append(args, "--disable-parent-config-comments="+strconv.FormatBool(cfg.DisableParentConfigComments))
Expand Down
Loading

0 comments on commit 3b21783

Please sign in to comment.