github/overleaf
1
0
Fork 0
mirror of https://github.com/overleaf/overleaf.git synced 2024-11-21 20:47:08 -05:00
Code Issues Projects Releases Packages Wiki Activity

Import apt version 2.3.8

Browse source
This commit is contained in:
James Allen 2014-03-30 16:51:48 +01:00
parent c24cc69136
commit 0e7b9d3ab1
20 changed files with 1899 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

173
server-ce/chef/cookbooks/apt/CHANGELOG.md Normal file
View file

@ -0,0 +1,173 @@
apt Cookbook CHANGELOG
======================
This file is used to list changes made in each version of the apt cookbook.
v2.3.8 (2014-02-14)
-------------------
### Bug
- **[COOK-4287](https://tickets.opscode.com/browse/COOK-4287)** - Cleanup the Kitchen
v2.3.6
------
* [COOK-4154] - Add chefspec matchers.rb file to apt cookbook
* [COOK-4102] - Only index created repository
v2.3.6
------
* [COOK-4154] - Add chefspec matchers.rb file to apt cookbook
* [COOK-4102] - Only index created repository
v2.3.4
------
No change. Version bump for toolchain sanity
v2.3.2
------
- [COOK-3905] apt-get-update-periodic: configuration for the update period
- Updating style for rubocops
- Updating test-kitchen harness
v2.3.0
------
### Bug
- **[COOK-3812](https://tickets.opscode.com/browse/COOK-3812)** - Add a way to bypass the apt existence check
### Improvement
- **[COOK-3567](https://tickets.opscode.com/browse/COOK-3567)** - Allow users to bypass apt-cache via attributes
v2.2.1
------
### Improvement
- **[COOK-664](https://tickets.opscode.com/browse/COOK-664)** - Check platform before running apt-specific commands
v2.2.0
------
### Bug
- **[COOK-3707](https://tickets.opscode.com/browse/COOK-3707)** - multiple nics confuse apt::cacher-client
v2.1.2
------
### Improvement
- **[COOK-3551](https://tickets.opscode.com/browse/COOK-3551)** - Allow user to set up a trusted APT repository
v2.1.1
------
### Bug
- **[COOK-1856](https://tickets.opscode.com/browse/COOK-1856)** - Match GPG keys without case sensitivity
v2.1.0
------
- [COOK-3426]: cacher-ng fails with restrict_environment set to true
- [COOK-2859]: cacher-client executes out of order
- [COOK-3052]: Long GPG keys are downloaded on every run
- [COOK-1856]: apt cookbook should match keys without case sensitivity
- [COOK-3255]: Attribute name incorrect in README
- [COOK-3225]: Call use_inline_resources only if defined
- [COOK-3386]: Cache dir for apt-cacher-ng
- [COOK-3291]: apt_repository: enable usage of a keyserver on port 80
- Greatly expanded test coverage with ChefSpec and Test-Kitchen
v2.0.0
------
### Bug
- [COOK-2258]: apt: LWRP results in error under why-run mode in apt 1.9.0 cookbook
v1.10.0
-------
### Improvement
- [COOK-2885]: Improvements for apt cache server search
### Bug
- [COOK-2441]: Apt recipe broken in new chef version
- [COOK-2660]: Create Debian 6.0 "squeeze" specific template for
apt-cacher-ng
v1.9.2
------
- [COOK-2631] - Create Ubuntu 10.04 specific template for apt-cacher-ng
v1.9.0
------
- [COOK-2185] - Proxy for apt-key
- [COOK-2338] - Support pinning by glob() or regexp
v1.8.4
------
- [COOK-2171] - Update README to clarify required Chef version: 10.18.0
or higher.
v1.8.2
------
- [COOK-2112] - need [] around "arch" in sources.list entries
- [COOK-2171] - fixes a regression in the notification
v1.8.0
------
- [COOK-2143] - Allow for a custom cacher-ng port
- [COOK-2171] - On `apt_repository.run_action(:add)` the source file
is not created.
- [COOK-2184] - apt::cacher-ng, use `cacher_port` attribute in
acng.conf
v1.7.0
------
- [COOK-2082] - add "arch" parameter to apt_repository LWRP
v1.6.0
------
- [COOK-1893] - `apt_preference` use "`package_name`" resource instead of "name"
- [COOK-1894] - change filename for sources.list.d files
- [COOK-1914] - Wrong dir permissions for /etc/apt/preferences.d/
- [COOK-1942] - README.md has wrong name for the keyserver attribute
- [COOK-2019] - create 01proxy before any other apt-get updates get executed
v1.5.2
------
- [COOK-1682] - use template instead of file resource in apt::cacher-client
- [COOK-1875] - cacher-client should be Environment-aware
V1.5.0
------
- [COOK-1500] - Avoid triggering apt-get update
- [COOK-1548] - Add execute commands for autoclean and autoremove
- [COOK-1591] - Setting up the apt proxy should leave https
connections direct
- [COOK-1596] - execute[apt-get-update-periodic] never runs
- [COOK-1762] - create /etc/apt/preferences.d directory
- [COOK-1776] - apt key check isn't idempotent
v1.4.8
------
* Adds test-kitchen support
- [COOK-1435] - repository lwrp is not idempotent with http key
v1.4.6
------
- [COOK-1530] - apt_repository isn't aware of update-success-stamp
file (also reverts COOK-1382 patch).
v1.4.4
------
- [COOK-1229] - Allow cacher IP to be set manually in non-Chef Solo
environments
- [COOK-1530] - Immediately update apt-cache when sources.list file is dropped off
v1.4.2
------
- [COOK-1155] - LWRP for apt pinning
v1.4.0
------
- [COOK-889] - overwrite existing repo source files
- [COOK-921] - optionally use cookbook\_file or remote\_file for key
- [COOK-1032] - fixes problem with apt repository key installation

248
server-ce/chef/cookbooks/apt/README.md Normal file
View file

@ -0,0 +1,248 @@
apt Cookbook
============
This cookbook includes recipes to execute apt-get update to ensure the local APT package cache is up to date. There are recipes for managing the apt-cacher-ng caching proxy and proxy clients. It also includes a LWRP for managing APT repositories in /etc/apt/sources.list.d as well as an LWRP for pinning packages via /etc/apt/preferences.d.
Requirements
------------
**Version 2.0.0+ of this cookbook requires Chef 11.0.0 or later**. If your Chef version is earlier than 11.0.0, use version 1.10.0 of this cookbook.
Version 1.8.2 to 1.10.0 of this cookbook requires **Chef 10.16.4** or later.
If your Chef version is earlier than 10.16.4, use version 1.7.0 of this cookbook.
### Platform
Please refer to the [TESTING file](TESTING.md) to see the currently (and passing) tested platforms. The release was tested on:
* Ubuntu 10.04
* Ubuntu 12.04
* Ubuntu 13.04
* Debian 7.1
* Debian 6.0 (have with manual testing)
May work with or without modification on other Debian derivatives.
-------
### default
This recipe installs the `update-notifier-common` package to provide the timestamp file used to only run `apt-get update` if the cache is more than one day old.
This recipe should appear first in the run list of Debian or Ubuntu nodes to ensure that the package cache is up to date before managing any `package` resources with Chef.
This recipe also sets up a local cache directory for preseeding packages.
**Including the default recipe on a node that does not support apt (such as Windows) results in a noop.**
### cacher-client
Configures the node to use the `apt-cacher-ng` server as a client.
#### Bypassing the cache
Occasionally you may come across repositories that do not play nicely when the node is using an `apt-cacher-ng` server. You can configure `cacher-client` to bypass the server and connect directly to the repository with the `cache_bypass` attribute.
To do this, you need to override the `cache_bypass` attribute with an array of repositories, with each array key as the repository URL and value as the protocol to use:
```json
{
...,
'apt': {
...,
'cache_bypass': {
URL: PROTOCOL
}
}
}
```
For example, to prevent caching and directly connect to the repository at `download.oracle.com` via http:
```json
{
'apt': {
'cache_bypass': {
'download.oracle.com': 'http'
}
}
}
```
### cacher-ng
Installs the `apt-cacher-ng` package and service so the system can provide APT caching. You can check the usage report at http://{hostname}:3142/acng-report.html.
If you wish to help the `cacher-ng` recipe seed itself, you must now explicitly include the `cacher-client` recipe in your run list **after** `cacher-ng` or you will block your ability to install any packages (ie. `apt-cacher-ng`).
Attributes
----------
* `['apt']['cacher_ipaddress']` - use a cacher server (or standard proxy server) not available via search
* `['apt']['cacher_interface]` - interface to connect to the cacher-ng service, no default.
* `['apt']['cacher_port']` - port for the cacher-ng service (either client or server), default is '3142'
* `['apt']['cacher_dir']` - directory used by cacher-ng service, default is '/var/cache/apt-cacher-ng'
* `['apt']['cacher-client']['restrict_environment']` - restrict your node to using the `apt-cacher-ng` server in your Environment, default is 'false'
* `['apt']['compiletime']` - force the `cacher-client` recipe to run before other recipes. It forces apt to use the proxy before other recipes run. Useful if your nodes have limited access to public apt repositories. This is overridden if the `cacher-ng` recipe is in your run list. Default is 'false'
* `['apt']['cache_bypass']` - array of URLs to bypass the cache. Accepts the URL and protocol to fetch directly from the remote repository and not attempt to cache
* `['apt']['periodic_update_min_delay']` - minimum delay (in seconds) beetween two actual executions of `apt-get update` by the `execute[apt-get-update-periodic]` resource, default is '86400' (24 hours)
Libraries
---------
There is an `interface_ipaddress` method that returns the IP address for a particular host and interface, used by the `cacher-client` recipe. To enable it on the server use the `['apt']['cacher_interface']` attribute.
Resources/Providers
-------------------
### `apt_repository`
This LWRP provides an easy way to manage additional APT repositories. Adding a new repository will notify running the `execute[apt-get-update]` resource immediately.
#### Actions
- :add: creates a repository file and builds the repository listing
- :remove: removes the repository file
#### Attribute Parameters
- repo_name: name attribute. The name of the channel to discover
- uri: the base of the Debian distribution
- distribution: this is usually your release's codename...ie something like `karmic`, `lucid` or `maverick`
- components: package groupings..when it doubt use `main`
- arch: constrain package to a particular arch like `i386`, `amd64` or even `armhf` or `powerpc`. Defaults to nil.
- trusted: treat all packages from this repository as authenticated regardless of signature
- deb_src: whether or not to add the repository as a source repo as well - value can be `true` or `false`, default `false`.
- keyserver: the GPG keyserver where the key for the repo should be retrieved
- key: if a `keyserver` is provided, this is assumed to be the fingerprint, otherwise it can be either the URI to the GPG key for the repo, or a cookbook_file.
- key_proxy: if set, pass the specified proxy via `http-proxy=` to GPG.
- cookbook: if key should be a cookbook_file, specify a cookbook where the key is located for files/default. Defaults to nil, so it will use the cookbook where the resource is used.
#### Examples
Add the Zenoss repo:
```ruby
apt_repository 'zenoss' do
uri 'http://dev.zenoss.org/deb'
components ['main', 'stable']
end
```
Add the Nginx PPA, grabbing the key from keyserver:
```ruby
apt_repository 'nginx-php' do
uri 'http://ppa.launchpad.net/nginx/php5/ubuntu'
distribution node['lsb']['codename']
components ['main']
keyserver 'keyserver.ubuntu.com'
key 'C300EE8C'
end
```
Add the Nginx PPA, grab the key from the keyserver, and add source repo:
```ruby
apt_repository 'nginx-php' do
uri 'http://ppa.launchpad.net/nginx/php5/ubuntu'
distribution node['lsb']['codename']
components ['main']
keyserver 'keyserver.ubuntu.com'
key 'C300EE8C'
deb_src true
end
```
Add the Cloudera Repo of CDH4 packages for Ubuntu 12.04 on AMD64:
```ruby
apt_repository 'cloudera' do
uri 'http://archive.cloudera.com/cdh4/ubuntu/precise/amd64/cdh'
arch 'amd64'
distribution 'precise-cdh4'
components ['contrib']
key 'http://archive.cloudera.com/debian/archive.key'
end
```
Remove Zenoss repo:
```ruby
apt_repository 'zenoss' do
action :remove
end
```
### `apt_preference`
This LWRP provides an easy way to pin packages in /etc/apt/preferences.d. Although apt-pinning is quite helpful from time to time please note that Debian does not encourage its use without thorough consideration.
Further information regarding apt-pinning is available via http://wiki.debian.org/AptPreferences.
#### Actions
- :add: creates a preferences file under /etc/apt/preferences.d
- :remove: Removes the file, therefore unpin the package
#### Attribute Parameters
- package_name: name attribute. The name of the package
- glob: Pin by glob() expression or regexp surrounded by /.
- pin: The package version/repository to pin
- pin_priority: The pinning priority aka "the highest package version wins"
#### Examples
Pin libmysqlclient16 to version 5.1.49-3:
```ruby
apt_preference 'libmysqlclient16' do
pin 'version 5.1.49-3'
pin_priority '700'
end
```
Unpin libmysqlclient16:
```ruby
apt_preference 'libmysqlclient16' do
action :remove
end
```
Pin all packages from dotdeb.org:
```ruby
apt_preference 'dotdeb' do
glob '*'
pin 'origin packages.dotdeb.org'
pin_priority '700'
end
```
Usage
-----
Put `recipe[apt]` first in the run list. If you have other recipes that you want to use to configure how apt behaves, like new sources, notify the execute resource to run, e.g.:
```ruby
template '/etc/apt/sources.list.d/my_apt_sources.list' do
notifies :run, 'execute[apt-get update]', :immediately
end
```
The above will run during execution phase since it is a normal template resource, and should appear before other package resources that need the sources in the template.
Put `recipe[apt::cacher-ng]` in the run_list for a server to provide APT caching and add `recipe[apt::cacher-client]` on the rest of the Debian-based nodes to take advantage of the caching server.
If you want to cleanup unused packages, there is also the `apt-get autoclean` and `apt-get autoremove` resources provided for automated cleanup.
License & Authors
-----------------
- Author:: Joshua Timberman (joshua@opscode.com)
- Author:: Matt Ray (matt@opscode.com)
- Author:: Seth Chisamore (schisamo@opscode.com)
```text
Copyright 2009-2013, Opscode, Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
```

28
server-ce/chef/cookbooks/apt/attributes/default.rb Normal file
View file

@ -0,0 +1,28 @@
#
# Cookbook Name:: apt
# Attributes:: default
#
# Copyright 2009-2013, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
default['apt']['cacher-client']['restrict_environment'] = false
default['apt']['cacher_dir'] = '/var/cache/apt-cacher-ng'
default['apt']['cacher_interface'] = nil
default['apt']['cacher_port'] = 3142
default['apt']['caching_server'] = false
default['apt']['compiletime'] = false
default['apt']['key_proxy'] = ''
default['apt']['cache_bypass'] = {}
default['apt']['periodic_update_min_delay'] = 86_400

50
server-ce/chef/cookbooks/apt/files/default/apt-proxy-v2.conf Normal file
View file

@ -0,0 +1,50 @@
[DEFAULT]
;; All times are in seconds, but you can add a suffix
;; for minutes(m), hours(h) or days(d)
;; commented out address so apt-proxy will listen on all IPs
;; address = 127.0.0.1
port = 9999
cache_dir = /var/cache/apt-proxy
;; Control files (Packages/Sources/Contents) refresh rate
min_refresh_delay = 1s
complete_clientless_downloads = 1
;; Debugging settings.
debug = all:4 db:0
time = 30
passive_ftp = on
;;--------------------------------------------------------------
;; Cache housekeeping
cleanup_freq = 1d
max_age = 120d
max_versions = 3
;;---------------------------------------------------------------
;; Backend servers
;;
;; Place each server in its own [section]
[ubuntu]
; Ubuntu archive
backends =
http://us.archive.ubuntu.com/ubuntu
[ubuntu-security]
; Ubuntu security updates
backends = http://security.ubuntu.com/ubuntu
[debian]
;; Backend servers, in order of preference
backends =
http://debian.osuosl.org/debian/
[security]
;; Debian security archive
backends =
http://security.debian.org/debian-security
http://ftp2.de.debian.org/debian-security

48
server-ce/chef/cookbooks/apt/libraries/helpers.rb Normal file
View file

@ -0,0 +1,48 @@
#
# Cookbook Name:: apt
# Library:: helpers
#
# Copyright 2013 Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
module Apt
# Helpers for apt
module Helpers
# Determines if apt is installed on a system.
#
# @return [Boolean]
def apt_installed?
!which('apt-get').nil?
end
# Finds a command in $PATH
#
# @return [String, nil]
def which(cmd)
paths = (ENV['PATH'].split(::File::PATH_SEPARATOR) + %w(/bin /usr/bin /sbin /usr/sbin))
paths.each do |path|
possible = File.join(path, cmd)
return possible if File.executable?(possible)
end
nil
end
end
end
Chef::Recipe.send(:include, ::Apt::Helpers)
Chef::Resource.send(:include, ::Apt::Helpers)
Chef::Provider.send(:include, ::Apt::Helpers)

17
server-ce/chef/cookbooks/apt/libraries/matchers.rb Normal file
View file

@ -0,0 +1,17 @@
if defined?(ChefSpec)
def add_apt_preference(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:apt_preference, :add, resource_name)
end
def remove_apt_preference(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:apt_preference, :remove, resource_name)
end
def add_apt_repository(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:apt_repository, :add, resource_name)
end
def remove_apt_repository(resource_name)
ChefSpec::Matchers::ResourceMatcher.new(:apt_repository, :remove, resource_name)
end
end

31
server-ce/chef/cookbooks/apt/libraries/network.rb Normal file
View file

@ -0,0 +1,31 @@
#
# Cookbook Name:: apt
# library:: network
#
# Copyright 2013, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
module ::Apt
def interface_ipaddress(host, interface)
if interface
addresses = host['network']['interfaces'][interface]['addresses']
addresses.select do |ip, data|
return ip if data['family'].eql?('inet')
end
else
return host.ipaddress
end
end
end

54
server-ce/chef/cookbooks/apt/metadata.json Normal file
View file

File diff suppressed because one or more lines are too long

34
server-ce/chef/cookbooks/apt/metadata.rb Normal file
View file

@ -0,0 +1,34 @@
name 'apt'
maintainer 'Opscode, Inc.'
maintainer_email 'cookbooks@opscode.com'
license 'Apache 2.0'
description 'Configures apt and apt services and LWRPs for managing apt repositories and preferences'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version '2.3.8'
recipe 'apt', 'Runs apt-get update during compile phase and sets up preseed directories'
recipe 'apt::cacher-ng', 'Set up an apt-cacher-ng caching proxy'
recipe 'apt::cacher-client', 'Client for the apt::cacher-ng caching proxy'
%w{ ubuntu debian }.each do |os|
supports os
end
attribute 'apt/cacher-client/restrict_environment',
:description => 'Whether to restrict the search for the caching server to the same environment as this node',
:default => 'false'
attribute 'apt/cacher_port',
:description => 'Default listen port for the caching server',
:default => '3142'
attribute 'apt/cacher_interface',
:description => 'Default listen interface for the caching server',
:default => nil
attribute 'apt/key_proxy',
:description => 'Passed as the proxy passed to GPG for the apt_repository resource',
:default => ''
attribute 'apt/caching_server',
:description => 'Set this to true if the node is a caching server',
:default => 'false'

63
server-ce/chef/cookbooks/apt/providers/preference.rb Normal file
View file

@ -0,0 +1,63 @@
#
# Cookbook Name:: apt
# Provider:: preference
#
# Copyright 2010-2011, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Build preferences.d file contents
def build_pref(package_name, pin, pin_priority)
"Package: #{package_name}\nPin: #{pin}\nPin-Priority: #{pin_priority}\n"
end
action :add do
new_resource.updated_by_last_action(false)
preference = build_pref(
new_resource.glob || new_resource.package_name,
new_resource.pin,
new_resource.pin_priority
)
preference_dir = directory '/etc/apt/preferences.d' do
owner 'root'
group 'root'
mode 00755
recursive true
action :nothing
end
preference_file = file "/etc/apt/preferences.d/#{new_resource.name}" do
owner 'root'
group 'root'
mode 00644
content preference
action :nothing
end
preference_dir.run_action(:create)
# write out the preference file, replace it if it already exists
preference_file.run_action(:create)
end
action :remove do
if ::File.exists?("/etc/apt/preferences.d/#{new_resource.name}")
Chef::Log.info "Un-pinning #{new_resource.name} from /etc/apt/preferences.d/"
file "/etc/apt/preferences.d/#{new_resource.name}" do
action :delete
end
new_resource.updated_by_last_action(true)
end
end

150
server-ce/chef/cookbooks/apt/providers/repository.rb Normal file
View file

@ -0,0 +1,150 @@
#
# Cookbook Name:: apt
# Provider:: repository
#
# Copyright 2010-2011, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
use_inline_resources if defined?(use_inline_resources)
def whyrun_supported?
true
end
# install apt key from keyserver
def install_key_from_keyserver(key, keyserver)
execute "install-key #{key}" do
if !node['apt']['key_proxy'].empty?
command "apt-key adv --keyserver-options http-proxy=#{node['apt']['key_proxy']} --keyserver hkp://#{keyserver}:80 --recv #{key}"
else
command "apt-key adv --keyserver #{keyserver} --recv #{key}"
end
action :run
not_if do
extract_fingerprints_from_cmd('apt-key finger').any? do |fingerprint|
fingerprint.end_with?(key.upcase)
end
end
end
end
# run command and extract gpg ids
def extract_fingerprints_from_cmd(cmd)
so = Mixlib::ShellOut.new(cmd)
so.run_command
so.stdout.split(/\n/).map do |t|
if z = t.match(/^ +Key fingerprint = ([0-9A-F ]+)/)
z[1].split.join
end
end.compact
end
# install apt key from URI
def install_key_from_uri(uri)
key_name = uri.split(/\//).last
cached_keyfile = "#{Chef::Config[:file_cache_path]}/#{key_name}"
if new_resource.key =~ /http/
remote_file cached_keyfile do
source new_resource.key
mode 00644
action :create
end
else
cookbook_file cached_keyfile do
source new_resource.key
cookbook new_resource.cookbook
mode 00644
action :create
end
end
execute "install-key #{key_name}" do
command "apt-key add #{cached_keyfile}"
action :run
not_if do
installed_keys = extract_fingerprints_from_cmd('apt-key finger')
proposed_keys = extract_fingerprints_from_cmd("gpg --with-fingerprint #{cached_keyfile}")
(installed_keys & proposed_keys).sort == proposed_keys.sort
end
end
end
# build repo file contents
def build_repo(uri, distribution, components, trusted, arch, add_deb_src)
components = components.join(' ') if components.respond_to?(:join)
repo_options = []
repo_options << "arch=#{arch}" if arch
repo_options << 'trusted=yes' if trusted
repo_options = '[' + repo_options.join(' ') + ']' unless repo_options.empty?
repo_info = "#{uri} #{distribution} #{components}\n"
repo_info = "#{repo_options} #{repo_info}" unless repo_options.empty?
repo = "deb #{repo_info}"
repo << "deb-src #{repo_info}" if add_deb_src
repo
end
action :add do
# add key
if new_resource.keyserver && new_resource.key
install_key_from_keyserver(new_resource.key, new_resource.keyserver)
elsif new_resource.key
install_key_from_uri(new_resource.key)
end
file '/var/lib/apt/periodic/update-success-stamp' do
action :nothing
end
execute 'apt-cache gencaches' do
ignore_failure true
action :nothing
end
execute 'apt-get update' do
command "apt-get update -o Dir::Etc::sourcelist='sources.list.d/#{new_resource.name}.list' -o Dir::Etc::sourceparts='-' -o APT::Get::List-Cleanup='0'"
ignore_failure true
action :nothing
notifies :run, 'execute[apt-cache gencaches]', :immediately
end
# build repo file
repository = build_repo(
new_resource.uri,
new_resource.distribution,
new_resource.components,
new_resource.trusted,
new_resource.arch,
new_resource.deb_src
)
file "/etc/apt/sources.list.d/#{new_resource.name}.list" do
owner 'root'
group 'root'
mode 00644
content repository
action :create
notifies :delete, 'file[/var/lib/apt/periodic/update-success-stamp]', :immediately
notifies :run, 'execute[apt-get update]', :immediately if new_resource.cache_rebuild
end
end
action :remove do
if ::File.exists?("/etc/apt/sources.list.d/#{new_resource.name}.list")
Chef::Log.info "Removing #{new_resource.name} repository from /etc/apt/sources.list.d/"
file "/etc/apt/sources.list.d/#{new_resource.name}.list" do
action :delete
end
end
end

81
server-ce/chef/cookbooks/apt/recipes/cacher-client.rb Normal file
View file

@ -0,0 +1,81 @@
#
# Cookbook Name:: apt
# Recipe:: cacher-client
#
# Copyright 2011-2013 Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
class ::Chef::Recipe
include ::Apt
end
# remove Acquire::http::Proxy lines from /etc/apt/apt.conf since we use 01proxy
# these are leftover from preseed installs
execute 'Remove proxy from /etc/apt/apt.conf' do
command "sed --in-place '/^Acquire::http::Proxy/d' /etc/apt/apt.conf"
only_if 'grep Acquire::http::Proxy /etc/apt/apt.conf'
end
servers = []
if node['apt']
if node['apt']['cacher_ipaddress']
cacher = Chef::Node.new
cacher.default.name = node['apt']['cacher_ipaddress']
cacher.default.ipaddress = node['apt']['cacher_ipaddress']
cacher.default.apt.cacher_port = node['apt']['cacher_port']
cacher.default.apt_cacher_interface = node['apt']['cacher_interface']
servers << cacher
elsif node['apt']['caching_server']
node.override['apt']['compiletime'] = false
servers << node
end
end
unless Chef::Config[:solo] || servers.length > 0
query = 'apt_caching_server:true'
query += " AND chef_environment:#{node.chef_environment}" if node['apt']['cacher-client']['restrict_environment']
Chef::Log.debug("apt::cacher-client searching for '#{query}'")
servers += search(:node, query)
end
if servers.length > 0
Chef::Log.info("apt-cacher-ng server found on #{servers[0]}.")
if servers[0]['apt']['cacher_interface']
cacher_ipaddress = interface_ipaddress(servers[0], servers[0]['apt']['cacher_interface'])
else
cacher_ipaddress = servers[0].ipaddress
end
t = template '/etc/apt/apt.conf.d/01proxy' do
source '01proxy.erb'
owner 'root'
group 'root'
mode 00644
variables(
:proxy => cacher_ipaddress,
:port => servers[0]['apt']['cacher_port'],
:bypass => node['apt']['cache_bypass']
)
action(node['apt']['compiletime'] ? :nothing : :create)
notifies :run, 'execute[apt-get update]', :immediately
end
t.run_action(:create) if node['apt']['compiletime']
else
Chef::Log.info('No apt-cacher-ng server found.')
file '/etc/apt/apt.conf.d/01proxy' do
action :delete
end
end
include_recipe 'apt::default'

43
server-ce/chef/cookbooks/apt/recipes/cacher-ng.rb Normal file
View file

@ -0,0 +1,43 @@
#
# Cookbook Name:: apt
# Recipe:: cacher-ng
#
# Copyright 2008-2013, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the 'License');
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an 'AS IS' BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
node.set['apt']['caching_server'] = true
package 'apt-cacher-ng' do
action :install
end
directory node['apt']['cacher_dir'] do
owner 'apt-cacher-ng'
group 'apt-cacher-ng'
mode 0755
end
template '/etc/apt-cacher-ng/acng.conf' do
source 'acng.conf.erb'
owner 'root'
group 'root'
mode 00644
notifies :restart, 'service[apt-cacher-ng]', :immediately
end
service 'apt-cacher-ng' do
supports :restart => true, :status => false
action [:enable, :start]
end

82
server-ce/chef/cookbooks/apt/recipes/default.rb Normal file
View file

@ -0,0 +1,82 @@
#
# Cookbook Name:: apt
# Recipe:: default
#
# Copyright 2008-2013, Opscode, Inc.
# Copyright 2009, Bryan McLellan <btm@loftninjas.org>
#
# Licensed under the Apache License, Version 2.0 (the 'License');
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an 'AS IS' BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# On systems where apt is not installed, the resources in this recipe are not
# executed. However, they _must_ still be present in the resource collection
# or other cookbooks which notify these resources will fail on non-apt-enabled
# systems.
Chef::Log.debug 'apt is not installed. Apt-specific resources will not be executed.' unless apt_installed?
# Run apt-get update to create the stamp file
execute 'apt-get-update' do
command 'apt-get update'
ignore_failure true
only_if { apt_installed? }
not_if { ::File.exists?('/var/lib/apt/periodic/update-success-stamp') }
end
# For other recipes to call to force an update
execute 'apt-get update' do
command 'apt-get update'
ignore_failure true
only_if { apt_installed? }
action :nothing
end
# Automatically remove packages that are no longer needed for dependencies
execute 'apt-get autoremove' do
command 'apt-get -y autoremove'
only_if { apt_installed? }
action :nothing
end
# Automatically remove .deb files for packages no longer on your system
execute 'apt-get autoclean' do
command 'apt-get -y autoclean'
only_if { apt_installed? }
action :nothing
end
# provides /var/lib/apt/periodic/update-success-stamp on apt-get update
package 'update-notifier-common' do
notifies :run, 'execute[apt-get-update]', :immediately
only_if { apt_installed? }
end
execute 'apt-get-update-periodic' do
command 'apt-get update'
ignore_failure true
only_if do
apt_installed? &&
::File.exists?('/var/lib/apt/periodic/update-success-stamp') &&
::File.mtime('/var/lib/apt/periodic/update-success-stamp') < Time.now - node['apt']['periodic_update_min_delay']
end
end
%w{/var/cache/local /var/cache/local/preseeding}.each do |dirname|
directory dirname do
owner 'root'
group 'root'
mode 00755
action :create
only_if { apt_installed? }
end
end

32
server-ce/chef/cookbooks/apt/resources/preference.rb Normal file
View file

@ -0,0 +1,32 @@
#
# Cookbook Name:: apt
# Resource:: preference
#
# Copyright 2010-2013, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
actions :add, :remove
default_action :add if defined?(default_action) # Chef > 10.8
# Needed for Chef versions < 0.10.10
def initialize(*args)
super
@action = :add
end
attribute :package_name, :kind_of => String, :name_attribute => true
attribute :glob, :kind_of => String
attribute :pin, :kind_of => String
attribute :pin_priority, :kind_of => String

43
server-ce/chef/cookbooks/apt/resources/repository.rb Normal file
View file

@ -0,0 +1,43 @@
#
# Cookbook Name:: apt
# Resource:: repository
#
# Copyright 2010-2013, Opscode, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
actions :add, :remove
default_action :add if defined?(default_action) # Chef > 10.8
# Needed for Chef versions < 0.10.10
def initialize(*args)
super
@action = :add
end
# name of the repo, used for source.list filename
attribute :repo_name, :kind_of => String, :name_attribute => true
attribute :uri, :kind_of => String
attribute :distribution, :kind_of => String
attribute :components, :kind_of => Array, :default => []
attribute :arch, :kind_of => String, :default => nil
attribute :trusted, :kind_of => [TrueClass, FalseClass], :default => false
# whether or not to add the repository as a source repo as well
attribute :deb_src, :default => false
attribute :keyserver, :kind_of => String, :default => nil
attribute :key, :kind_of => String, :default => nil
attribute :cookbook, :kind_of => String, :default => nil
# trigger cache rebuild
# If not you can trigger in the recipe itself after checking the status of resource.updated{_by_last_action}?
attribute :cache_rebuild, :kind_of => [TrueClass, FalseClass], :default => true

173
server-ce/chef/cookbooks/apt/templates/debian-6.0/acng.conf.erb Normal file
View file

@ -0,0 +1,173 @@
# Letter case in directive names does not matter. Must be separated with colons.
# Valid boolean values are a zero number for false, non-zero numbers for true.
CacheDir: <%= node['apt']['cacher_dir'] %>
# set empty to disable logging
LogDir: /var/log/apt-cacher-ng
# TCP (http) port
# Set to 9999 to emulate apt-proxy
Port:<%= node['apt']['cacher_port'] %>
# Addresses or hostnames to listen on. Multiple addresses must be separated by
# spaces. Each entry must be associated with a local interface. DNS resolution
# is performed using getaddrinfo(3) for all available protocols (i.e. IPv4 and
# IPv6 if available).
#
# Default: not set, will listen on all interfaces.
#
# BindAddress: localhost 192.168.7.254 publicNameOnMainInterface
#Proxy: http://www-proxy.example.net:80
#proxy: http://username:proxypassword@proxy.example.net:3128
# Repository remapping. See manual for details.
# In this example, backends file is generated during package installation.
Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian
Remap-uburep: file:ubuntu_mirrors /ubuntu ; file:backends_ubuntu
Remap-debvol: file:debvol_mirror*.gz /debian-volatile ; file:backends_debvol
Remap-cygwin: file:cygwin_mirrors /cygwin # ; file:backends_cygwin # incomplete, please create this file
# Virtual page accessible in a web browser to see statistics and status
# information, i.e. under http://localhost:3142/acng-report.html
ReportPage: acng-report.html
# Socket file for accessing through local UNIX socket instead of TCP/IP. Can be
# used with inetd bridge or cron client.
# SocketPath:/var/run/apt-cacher-ng/socket
# Forces log file to be written to disk after every line when set to 1. Default
# is 0, buffer flush happens after client disconnects.
#
# (technically, this is an alias to the Debug option provided for convenience)
#
# UnbufferLogs: 0
# Set to 0 to store only type, time and transfer sizes.
# 1 -> client IP and relative local path are logged too
# VerboseLog: 1
# Don't detach from the console
# ForeGround: 0
# Store the pid of the daemon process therein
# PidFile: /var/run/apt-cacher-ng/pid
# Forbid outgoing connections, work around them or respond with 503 error
# offlinemode:0
# Forbid all downloads that don't run through preconfigured backends (.where)
#ForceManaged: 0
# Days before considering an unreferenced file expired (to be deleted).
# Warning: if the value is set too low and particular index files are not
# available for some days (mirror downtime) there is a risk of deletion of
# still usefull package files.
ExTreshold: 4
# Stop expiration when a critical problem appeared. Currently only failed
# refresh of an index file is considered as critical.
#
# WARNING: don't touch this option or set to a non-zero number.
# Anything else is DANGEROUS and may cause data loss.
#
# ExAbortOnProblems: 1
# Replace some Windows/DOS-FS incompatible chars when storing
# StupidFs: 0
# Experimental feature for apt-listbugs: pass-through SOAP requests and
# responses to/from bugs.debian.org. If not set, default is true if
# ForceManaged is enabled and false otherwise.
# ForwardBtsSoap: 1
# The daemon has a small cache for DNS data, to speed up resolution. The
# expiration time of the DNS entries can be configured in seconds.
# DnsCacheSeconds: 3600
# Don't touch the following values without good consideration!
#
# Max. count of connection threads kept ready (for faster response in the
# future). Should be a sane value between 0 and average number of connections,
# and depend on the amount of spare RAM.
# MaxStandbyConThreads: 8
#
# Hard limit of active thread count for incomming connections, i.e. operation
# is refused when this value is reached (below zero = unlimited).
# MaxConThreads: -1
#
#VfilePattern = (^|.*?/)(Index|Packages\.bz2|Packages\.gz|Packages|Release|Release\.gpg|Sources\.bz2|Sources\.gz|Sources|release|index\.db-.*\.gz|Contents-[^/]*\.gz|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|/meta-release[^/]*|Translation[^/]*\.bz2)$
#PfilePattern = .*(\.deb|\.rpm|\.dsc|\.tar\.gz\.gpg|\.tar\.gz|\.diff\.gz|\.diff\.bz2|\.jigdo|\.template|changelog|copyright|\.udeb|\.diff/.*\.gz|vmlinuz|initrd\.gz|(Devel)?ReleaseAnnouncement(\\?.*)?)$
# Whitelist for expiration, file types not to be removed even when being
# unreferenced. Default: same as VfilePattern which is a safe bed. When and
# only when the only used mirrors are official repositories (with working
# Release files) then it might be set to something more restrictive, like
# (^|.*?/)(Release|Release\.gpg|release|meta-release|Translation[^/]*\.bz2)$
#WfilePattern = (^|.*?/)(Index|Packages\.bz2|Packages\.gz|Packages|Release|Release\.gpg|Sources\.bz2|Sources\.gz|Sources|release|index\.db-.*\.gz|Contents-[^/]*\.gz|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|/meta-release[^/]*|Translation[^/]*\.bz2)$
# Higher modes only working with the debug version
# Warning, writes a lot into apt-cacher.err logfile
# Value overwrites UnbufferLogs setting (aliased)
# Debug:3
# Usually, general purpose proxies like Squid expose the IP adress of the
# client user to the remote server using the X-Forwarded-For HTTP header. This
# behaviour can be optionally turned on with the Expose-Origin option.
# ExposeOrigin: 0
# When logging the originating IP address, trust the information supplied by
# the client in the X-Forwarded-For header.
# LogSubmittedOrigin: 0
# The version string reported to the peer, to be displayed as HTTP client (and
# version) in the logs of the mirror.
# WARNING: some archives use this header to detect/guess capabilities of the
# client (i.e. redirection support) and change the behaviour accordingly, while
# ACNG might not support the expected features. Expect side effects.
#
# UserAgent: Yet Another HTTP Client/1.2.3p4
# In some cases the Import and Expiration tasks might create fresh volatile
# data for internal use by reconstructing them using patch files. This
# by-product might be recompressed with bzip2 and with some luck the resulting
# file becomes identical to the *.bz2 file on the server, usable for APT
# clients trying to fetch the full .bz2 compressed version. Injection of the
# generated files into the cache has however a disadvantage on underpowered
# servers: bzip2 compession can create high load on the server system and the
# visible download of the busy .bz2 files also becomes slower.
#
# RecompBz2: 0
# Network timeout for outgoing connections.
# NetworkTimeout: 60
# Sometimes it makes sense to not store the data in cache and just return the
# package data to client as it comes in. DontCache parameters can enable this
# behaviour for certain URL types. The tokens are extended regular expressions
# that URLs are matched against.
#
# DontCacheRequested is applied to the URL as it comes in from the client.
# Example: exclude packages built with kernel-package for x86
# DontCacheRequested: linux-.*_10\...\.Custo._i386
# Example usecase: exclude popular private IP ranges from caching
# DontCacheRequested: 192.168.0 ^10\..* 172.30
#
# DontCacheResolved is applied to URLs after mapping to the target server. If
# multiple backend servers are specified then it's only matched against the
# download link for the FIRST possible source (due to implementation limits).
# Example usecase: all Ubuntu stuff comes from a local mirror (specified as
# backend), don't cache it again:
# DontCacheResolved: ubuntumirror.local.net
#
# DontCache directive sets (overrides) both, DontCacheResolved and
# DontCacheRequested. Provided for convenience, see those directives for
# details.
#
# Default permission set of freshly created files and directories, as octal
# numbers (see chmod(1) for details).
# Can by limited by the umask value (see umask(2) for details) if it's set in
# the environment of the starting shell, e.g. in apt-cacher-ng init script or
# in its configuration file.
# DirPerms: 00755
# FilePerms: 00664

5
server-ce/chef/cookbooks/apt/templates/default/01proxy.erb Normal file
View file

@ -0,0 +1,5 @@
Acquire::http::Proxy "http://<%= @proxy %>:<%= @port %>";
Acquire::https::Proxy "DIRECT";
<% @bypass.each do |bypass, type| %>
Acquire::<%= type %>::Proxy::<%= bypass %> "DIRECT";
<% end %>

275
server-ce/chef/cookbooks/apt/templates/default/acng.conf.erb Normal file
View file

@ -0,0 +1,275 @@
# Letter case in directive names does not matter. Must be separated with colons.
# Valid boolean values are a zero number for false, non-zero numbers for true.
CacheDir: <%= node['apt']['cacher_dir'] %>
# set empty to disable logging
LogDir: /var/log/apt-cacher-ng
# place to look for additional configuration and resource files if they are not
# found in the configuration directory
# SupportDir: /usr/lib/apt-cacher-ng
# TCP (http) port
# Set to 9999 to emulate apt-proxy
Port:<%= node['apt']['cacher_port'] %>
# Addresses or hostnames to listen on. Multiple addresses must be separated by
# spaces. Each entry must be an exact local address which is associated with a
# local interface. DNS resolution is performed using getaddrinfo(3) for all
# available protocols (IPv4, IPv6, ...). Using a protocol specific format will
# create binding(s) only on protocol specific socket(s) (e.g. 0.0.0.0 will listen
# only to IPv4).
#
# Default: not set, will listen on all interfaces and protocols
#
# BindAddress: localhost 192.168.7.254 publicNameOnMainInterface
# The specification of another proxy which shall be used for downloads.
# Username and password are, and see manual for limitations.
#
#Proxy: http://www-proxy.example.net:80
#proxy: username:proxypassword@proxy.example.net:3128
# Repository remapping. See manual for details.
# In this example, some backends files might be generated during package
# installation using information collected on the system.
Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian # Debian Archives
Remap-uburep: file:ubuntu_mirrors /ubuntu ; file:backends_ubuntu # Ubuntu Archives
Remap-debvol: file:debvol_mirror*.gz /debian-volatile ; file:backends_debvol # Debian Volatile Archives
Remap-cygwin: file:cygwin_mirrors /cygwin # ; file:backends_cygwin # incomplete, please create this file or specify preferred mirrors here
Remap-sfnet: file:sfnet_mirrors # ; file:backends_sfnet # incomplete, please create this file or specify preferred mirrors here
Remap-alxrep: file:archlx_mirrors /archlinux # ; file:backend_archlx # Arch Linux
Remap-fedora: file:fedora_mirrors # Fedora Linux
Remap-epel: file:epel_mirrors # Fedora EPEL
Remap-slrep: file:sl_mirrors # Scientific Linux
# This is usually not needed for security.debian.org because it's always the
# same DNS hostname. However, it might be enabled in order to use hooks,
# ForceManaged mode or special flags in this context.
# Remap-secdeb: security.debian.org
# Virtual page accessible in a web browser to see statistics and status
# information, i.e. under http://localhost:3142/acng-report.html
ReportPage: acng-report.html
# Socket file for accessing through local UNIX socket instead of TCP/IP. Can be
# used with inetd bridge or cron client.
# SocketPath:/var/run/apt-cacher-ng/socket
# Forces log file to be written to disk after every line when set to 1. Default
# is 0, buffers are flushed when the client disconnects.
#
# (technically, alias to the Debug option, see its documentation for details)
#
# UnbufferLogs: 0
# Set to 0 to store only type, time and transfer sizes.
# 1 -> client IP and relative local path are logged too
# VerboseLog: 1
# Don't detach from the console
# ForeGround: 0
# Store the pid of the daemon process therein
# PidFile: /var/run/apt-cacher-ng/pid
# Forbid outgoing connections, work around them or respond with 503 error
# offlinemode:0
# Forbid all downloads that don't run through preconfigured backends (.where)
#ForceManaged: 0
# Days before considering an unreferenced file expired (to be deleted).
# Warning: if the value is set too low and particular index files are not
# available for some days (mirror downtime) there is a risk of deletion of
# still useful package files.
ExTreshold: 4
# Stop expiration when a critical problem appeared. Currently only failed
# refresh of an index file is considered as critical.
#
# WARNING: don't touch this option or set to zero.
# Anything else is DANGEROUS and may cause data loss.
#
# ExAbortOnProblems: 1
# Replace some Windows/DOS-FS incompatible chars when storing
# StupidFs: 0
# Experimental feature for apt-listbugs: pass-through SOAP requests and
# responses to/from bugs.debian.org. If not set, default is true if
# ForceManaged is enabled and false otherwise.
# ForwardBtsSoap: 1
# The daemon has a small cache for DNS data, to speed up resolution. The
# expiration time of the DNS entries can be configured in seconds.
# DnsCacheSeconds: 3600
# Don't touch the following values without good consideration!
#
# Max. count of connection threads kept ready (for faster response in the
# future). Should be a sane value between 0 and average number of connections,
# and depend on the amount of spare RAM.
# MaxStandbyConThreads: 8
#
# Hard limit of active thread count for incoming connections, i.e. operation
# is refused when this value is reached (below zero = unlimited).
# MaxConThreads: -1
#
# Pigeonholing files with regular expressions (static/volatile). Can be
# overriden here but not should not be done permanently because future update
# of default settings would not be applied later.
# VfilePattern = (^|.*?/)(Index|Packages(\.gz|\.bz2|\.lzma|\.xz)?|InRelease|Release|Release\.gpg|Sources(\.gz|\.bz2|\.lzma|\.xz)?|release|index\.db-.*\.gz|Contents-[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|/meta-release[^/]*|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|((setup|setup-legacy)(\.ini|\.bz2|\.hint)(\.sig)?)|mirrors\.lst|repo(index|md)\.xml(\.asc|\.key)?|directory\.yast|products|content(\.asc|\.key)?|media|filelists\.xml\.gz|filelists\.sqlite\.bz2|repomd\.xml|packages\.[a-zA-Z][a-zA-Z]\.gz|info\.txt|license\.tar\.gz|license\.zip|.*\.db(\.tar\.gz)?|.*\.files\.tar\.gz|.*\.abs\.tar\.gz|metalink\?repo|.*prestodelta\.xml\.gz)$|/dists/.*/installer-[^/]+/[^0-9][^/]+/images/.*
# PfilePattern = .*(\.d?deb|\.rpm|\.dsc|\.tar(\.gz|\.bz2|\.lzma|\.xz)(\.gpg)?|\.diff(\.gz|\.bz2|\.lzma|\.xz)|\.jigdo|\.template|changelog|copyright|\.udeb|\.debdelta|\.diff/.*\.gz|(Devel)?ReleaseAnnouncement(\?.*)?|[a-f0-9]+-(susedata|updateinfo|primary|deltainfo).xml.gz|fonts/(final/)?[a-z]+32.exe(\?download.*)?|/dists/.*/installer-[^/]+/[0-9][^/]+/images/.*)$
# Whitelist for expiration, file types not to be removed even when being
# unreferenced. Default: many parts from VfilePattern where no parent index
# exists or might be unknown.
# WfilePattern = (^|.*?/)(Release|InRelease|Release\.gpg|(Packages|Sources)(\.gz|\.bz2|\.lzma|\.xz)?|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|.*\.xml|.*\.db\.tar\.gz|.*\.files\.tar\.gz|.*\.abs\.tar\.gz|[a-z]+32.exe)$|/dists/.*/installer-.*/images/.*
# Higher modes only working with the debug version
# Warning, writes a lot into apt-cacher.err logfile
# Value overwrites UnbufferLogs setting (aliased)
# Debug:3
# Usually, general purpose proxies like Squid expose the IP address of the
# client user to the remote server using the X-Forwarded-For HTTP header. This
# behaviour can be optionally turned on with the Expose-Origin option.
# ExposeOrigin: 0
# When logging the originating IP address, trust the information supplied by
# the client in the X-Forwarded-For header.
# LogSubmittedOrigin: 0
# The version string reported to the peer, to be displayed as HTTP client (and
# version) in the logs of the mirror.
# WARNING: some archives use this header to detect/guess capabilities of the
# client (i.e. redirection support) and change the behaviour accordingly, while
# ACNG might not support the expected features. Expect side effects.
#
# UserAgent: Yet Another HTTP Client/1.2.3p4
# In some cases the Import and Expiration tasks might create fresh volatile
# data for internal use by reconstructing them using patch files. This
# by-product might be recompressed with bzip2 and with some luck the resulting
# file becomes identical to the *.bz2 file on the server, usable for APT
# clients trying to fetch the full .bz2 compressed version. Injection of the
# generated files into the cache has however a disadvantage on underpowered
# servers: bzip2 compression can create high load on the server system and the
# visible download of the busy .bz2 files also becomes slower.
#
# RecompBz2: 0
# Network timeout for outgoing connections.
# NetworkTimeout: 60
# Sometimes it makes sense to not store the data in cache and just return the
# package data to client as it comes in. DontCache parameters can enable this
# behaviour for certain URL types. The tokens are extended regular expressions
# that URLs are matched against.
#
# DontCacheRequested is applied to the URL as it comes in from the client.
# Example: exclude packages built with kernel-package for x86
# DontCacheRequested: linux-.*_10\...\.Custo._i386
# Example usecase: exclude popular private IP ranges from caching
# DontCacheRequested: 192.168.0 ^10\..* 172.30
#
# DontCacheResolved is applied to URLs after mapping to the target server. If
# multiple backend servers are specified then it's only matched against the
# download link for the FIRST possible source (due to implementation limits).
# Example usecase: all Ubuntu stuff comes from a local mirror (specified as
# backend), don't cache it again:
# DontCacheResolved: ubuntumirror.local.net
#
# DontCache directive sets (overrides) both, DontCacheResolved and
# DontCacheRequested. Provided for convenience, see those directives for
# details.
#
# Default permission set of freshly created files and directories, as octal
# numbers (see chmod(1) for details).
# Can by limited by the umask value (see umask(2) for details) if it's set in
# the environment of the starting shell, e.g. in apt-cacher-ng init script or
# in its configuration file.
# DirPerms: 00755
# FilePerms: 00664
#
#
# It's possible to use use apt-cacher-ng as a regular web server with limited
# feature set, i.e.
# including directory browsing and download of any file;
# excluding sorting, mime types/encodings, CGI execution, index page
# redirection and other funny things.
# To get this behavior, mappings between virtual directories and real
# directories on the server must be defined with the LocalDirs directive.
# Virtual and real dirs are separated by spaces, multiple pairs are separated
# by semi-colons. Real directories must be absolute paths.
# NOTE: Since the names of that key directories share the same namespace as
# repository names (see Remap-...) it's administrators job to avoid such
# collisions on them (unless created deliberately).
#
# LocalDirs: woo /data/debarchive/woody ; hamm /data/debarchive/hamm
# Precache a set of files referenced by specified index files. This can be used
# to create a partial mirror usable for offline work. There are certain limits
# and restrictions on the path specification, see manual for details. A list of
# (maybe) relevant index files could be retrieved via
# "apt-get --print-uris update" on a client machine.
#
# PrecacheFor: debrep/dists/unstable/*/source/Sources* debrep/dists/unstable/*/binary-amd64/Packages*
# Arbitrary set of data to append to request headers sent over the wire. Should
# be a well formated HTTP headers part including newlines (DOS style) which
# can be entered as escape sequences (\r\n).
# RequestAppendix: X-Tracking-Choice: do-not-track\r\n
# Specifies the IP protocol families to use for remote connections. Order does
# matter, first specified are considered first. Possible combinations:
# v6 v4
# v4 v6
# v6
# v4
# (empty or not set: use system default)
#
# ConnectProto: v6 v4
# Regular expiration algorithm finds package files which are no longer listed
# in any index file and removes them of them after a safety period.
# This option allows to keep more versions of a package in the cache after
# safety period is over.
# KeepExtraVersions: 1
# Optionally uses TCP access control provided by libwrap, see hosts_access(5)
# for details. Daemon name is apt-cacher-ng. Default if not set: decided on
# startup by looking for explicit mentioning of apt-cacher-ng in
# /etc/hosts.allow or /etc/hosts.deny files.
# UseWrap: 0
# If many machines from the same local network attempt to update index files
# (apt-get update) at nearly the same time, the known state of these index file
# is temporarily frozen and multiple requests receive the cached response
# without contacting the server. This parameter (in seconds) specifies the
# length of this period before the files are considered outdated.
# Setting it too low transfers more data and increases remote server load,
# setting it too high (more than a couple of minutes) increases the risk of
# delivering inconsistent responses to the clients.
# FreshIndexMaxAge: 27
# Usually the users are not allowed to specify custom TCP ports of remote
# mirrors in the requests, only the default HTTP port can be used (instead,
# proxy administrator can create Remap- rules with custom ports). This
# restriction can be disabled by specifying a list of allowed ports or 0 for
# any port.
#
# AllowUserPorts: 80
# Normally the HTTP redirection responses are forwarded to the original caller
# (i.e. APT) which starts a new download attempt from the new URL. This
# solution is ok for client configurations with proxy mode but doesn't work
# well with configurations using URL prefixes. To work around this the server
# can restart its own download with another URL. However, this might be used to
# circumvent download source policies by malicious users.
# The RedirMax option specifies how many such redirects the server should
# follow per request, 0 disables the internal redirection. If not set,
# default value is 0 if ForceManaged is used and 5 otherwise.
#
# RedirMax: 5

269
server-ce/chef/cookbooks/apt/templates/ubuntu-10.04/acng.conf.erb Normal file
View file

@ -0,0 +1,269 @@
# Letter case in directive names does not matter. Must be separated with colons.
# Valid boolean values are a zero number for false, non-zero numbers for true.
CacheDir: <%= node['apt']['cacher_dir'] %>
# set empty to disable logging
LogDir: /var/log/apt-cacher-ng
# place to look for additional configuration and resource files if they are not
# found in the configuration directory
# SupportDir: /usr/lib/apt-cacher-ng
# TCP (http) port
# Set to 9999 to emulate apt-proxy
Port:<%= node['apt']['cacher_port'] %>
# Addresses or hostnames to listen on. Multiple addresses must be separated by
# spaces. Each entry must be an exact local address which is associated with a
# local interface. DNS resolution is performed using getaddrinfo(3) for all
# available protocols (IPv4, IPv6, ...). Using a protocol specific format will
# create binding(s) only on protocol specific socket(s) (e.g. 0.0.0.0 will listen
# only to IPv4).
#
# Default: not set, will listen on all interfaces and protocols
#
# BindAddress: localhost 192.168.7.254 publicNameOnMainInterface
# The specification of another proxy which shall be used for downloads.
# Username and password are, and see manual for limitations.
#
#Proxy: http://www-proxy.example.net:80
#proxy: username:proxypassword@proxy.example.net:3128
# Repository remapping. See manual for details.
# In this example, some backends files might be generated during package
# installation using information collected on the system.
Remap-debrep: file:deb_mirror*.gz /debian ; file:backends_debian # Debian Archives
Remap-uburep: file:ubuntu_mirrors /ubuntu ; file:backends_ubuntu # Ubuntu Archives
Remap-debvol: file:debvol_mirror*.gz /debian-volatile ; file:backends_debvol # Debian Volatile Archives
# This is usually not needed for security.debian.org because it's always the
# same DNS hostname. However, it might be enabled in order to use hooks,
# ForceManaged mode or special flags in this context.
# Remap-secdeb: security.debian.org
# Virtual page accessible in a web browser to see statistics and status
# information, i.e. under http://localhost:3142/acng-report.html
ReportPage: acng-report.html
# Socket file for accessing through local UNIX socket instead of TCP/IP. Can be
# used with inetd bridge or cron client.
# SocketPath:/var/run/apt-cacher-ng/socket
# Forces log file to be written to disk after every line when set to 1. Default
# is 0, buffers are flushed when the client disconnects.
#
# (technically, alias to the Debug option, see its documentation for details)
#
# UnbufferLogs: 0
# Set to 0 to store only type, time and transfer sizes.
# 1 -> client IP and relative local path are logged too
# VerboseLog: 1
# Don't detach from the console
# ForeGround: 0
# Store the pid of the daemon process therein
# PidFile: /var/run/apt-cacher-ng/pid
# Forbid outgoing connections, work around them or respond with 503 error
# offlinemode:0
# Forbid all downloads that don't run through preconfigured backends (.where)
#ForceManaged: 0
# Days before considering an unreferenced file expired (to be deleted).
# Warning: if the value is set too low and particular index files are not
# available for some days (mirror downtime) there is a risk of deletion of
# still useful package files.
ExTreshold: 4
# Stop expiration when a critical problem appeared. Currently only failed
# refresh of an index file is considered as critical.
#
# WARNING: don't touch this option or set to zero.
# Anything else is DANGEROUS and may cause data loss.
#
# ExAbortOnProblems: 1
# Replace some Windows/DOS-FS incompatible chars when storing
# StupidFs: 0
# Experimental feature for apt-listbugs: pass-through SOAP requests and
# responses to/from bugs.debian.org. If not set, default is true if
# ForceManaged is enabled and false otherwise.
# ForwardBtsSoap: 1
# The daemon has a small cache for DNS data, to speed up resolution. The
# expiration time of the DNS entries can be configured in seconds.
# DnsCacheSeconds: 3600
# Don't touch the following values without good consideration!
#
# Max. count of connection threads kept ready (for faster response in the
# future). Should be a sane value between 0 and average number of connections,
# and depend on the amount of spare RAM.
# MaxStandbyConThreads: 8
#
# Hard limit of active thread count for incoming connections, i.e. operation
# is refused when this value is reached (below zero = unlimited).
# MaxConThreads: -1
#
# Pigeonholing files with regular expressions (static/volatile). Can be
# overriden here but not should not be done permanently because future update
# of default settings would not be applied later.
# VfilePattern = (^|.*?/)(Index|Packages(\.gz|\.bz2|\.lzma|\.xz)?|InRelease|Release|Release\.gpg|Sources(\.gz|\.bz2|\.lzma|\.xz)?|release|index\.db-.*\.gz|Contents-[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|pkglist[^/]*\.bz2|rclist[^/]*\.bz2|/meta-release[^/]*|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|((setup|setup-legacy)(\.ini|\.bz2|\.hint)(\.sig)?)|mirrors\.lst|repo(index|md)\.xml(\.asc|\.key)?|directory\.yast|products|content(\.asc|\.key)?|media|filelists\.xml\.gz|filelists\.sqlite\.bz2|repomd\.xml|packages\.[a-zA-Z][a-zA-Z]\.gz|info\.txt|license\.tar\.gz|license\.zip|.*\.db(\.tar\.gz)?|.*\.files\.tar\.gz|.*\.abs\.tar\.gz|metalink\?repo|.*prestodelta\.xml\.gz)$|/dists/.*/installer-[^/]+/[^0-9][^/]+/images/.*
# PfilePattern = .*(\.d?deb|\.rpm|\.dsc|\.tar(\.gz|\.bz2|\.lzma|\.xz)(\.gpg)?|\.diff(\.gz|\.bz2|\.lzma|\.xz)|\.jigdo|\.template|changelog|copyright|\.udeb|\.debdelta|\.diff/.*\.gz|(Devel)?ReleaseAnnouncement(\?.*)?|[a-f0-9]+-(susedata|updateinfo|primary|deltainfo).xml.gz|fonts/(final/)?[a-z]+32.exe(\?download.*)?|/dists/.*/installer-[^/]+/[0-9][^/]+/images/.*)$
# Whitelist for expiration, file types not to be removed even when being
# unreferenced. Default: many parts from VfilePattern where no parent index
# exists or might be unknown.
# WfilePattern = (^|.*?/)(Release|InRelease|Release\.gpg|(Packages|Sources)(\.gz|\.bz2|\.lzma|\.xz)?|Translation[^/]*(\.gz|\.bz2|\.lzma|\.xz)?|MD5SUMS|SHA1SUMS|.*\.xml|.*\.db\.tar\.gz|.*\.files\.tar\.gz|.*\.abs\.tar\.gz|[a-z]+32.exe)$|/dists/.*/installer-.*/images/.*
# Higher modes only working with the debug version
# Warning, writes a lot into apt-cacher.err logfile
# Value overwrites UnbufferLogs setting (aliased)
# Debug:3
# Usually, general purpose proxies like Squid expose the IP address of the
# client user to the remote server using the X-Forwarded-For HTTP header. This
# behaviour can be optionally turned on with the Expose-Origin option.
# ExposeOrigin: 0
# When logging the originating IP address, trust the information supplied by
# the client in the X-Forwarded-For header.
# LogSubmittedOrigin: 0
# The version string reported to the peer, to be displayed as HTTP client (and
# version) in the logs of the mirror.
# WARNING: some archives use this header to detect/guess capabilities of the
# client (i.e. redirection support) and change the behaviour accordingly, while
# ACNG might not support the expected features. Expect side effects.
#
# UserAgent: Yet Another HTTP Client/1.2.3p4
# In some cases the Import and Expiration tasks might create fresh volatile
# data for internal use by reconstructing them using patch files. This
# by-product might be recompressed with bzip2 and with some luck the resulting
# file becomes identical to the *.bz2 file on the server, usable for APT
# clients trying to fetch the full .bz2 compressed version. Injection of the
# generated files into the cache has however a disadvantage on underpowered
# servers: bzip2 compression can create high load on the server system and the
# visible download of the busy .bz2 files also becomes slower.
#
# RecompBz2: 0
# Network timeout for outgoing connections.
# NetworkTimeout: 60
# Sometimes it makes sense to not store the data in cache and just return the
# package data to client as it comes in. DontCache parameters can enable this
# behaviour for certain URL types. The tokens are extended regular expressions
# that URLs are matched against.
#
# DontCacheRequested is applied to the URL as it comes in from the client.
# Example: exclude packages built with kernel-package for x86
# DontCacheRequested: linux-.*_10\...\.Custo._i386
# Example usecase: exclude popular private IP ranges from caching
# DontCacheRequested: 192.168.0 ^10\..* 172.30
#
# DontCacheResolved is applied to URLs after mapping to the target server. If
# multiple backend servers are specified then it's only matched against the
# download link for the FIRST possible source (due to implementation limits).
# Example usecase: all Ubuntu stuff comes from a local mirror (specified as
# backend), don't cache it again:
# DontCacheResolved: ubuntumirror.local.net
#
# DontCache directive sets (overrides) both, DontCacheResolved and
# DontCacheRequested. Provided for convenience, see those directives for
# details.
#
# Default permission set of freshly created files and directories, as octal
# numbers (see chmod(1) for details).
# Can by limited by the umask value (see umask(2) for details) if it's set in
# the environment of the starting shell, e.g. in apt-cacher-ng init script or
# in its configuration file.
# DirPerms: 00755
# FilePerms: 00664
#
#
# It's possible to use use apt-cacher-ng as a regular web server with limited
# feature set, i.e.
# including directory browsing and download of any file;
# excluding sorting, mime types/encodings, CGI execution, index page
# redirection and other funny things.
# To get this behavior, mappings between virtual directories and real
# directories on the server must be defined with the LocalDirs directive.
# Virtual and real dirs are separated by spaces, multiple pairs are separated
# by semi-colons. Real directories must be absolute paths.
# NOTE: Since the names of that key directories share the same namespace as
# repository names (see Remap-...) it's administrators job to avoid such
# collisions on them (unless created deliberately).
#
# LocalDirs: woo /data/debarchive/woody ; hamm /data/debarchive/hamm
# Precache a set of files referenced by specified index files. This can be used
# to create a partial mirror usable for offline work. There are certain limits
# and restrictions on the path specification, see manual for details. A list of
# (maybe) relevant index files could be retrieved via
# "apt-get --print-uris update" on a client machine.
#
# PrecacheFor: debrep/dists/unstable/*/source/Sources* debrep/dists/unstable/*/binary-amd64/Packages*
# Arbitrary set of data to append to request headers sent over the wire. Should
# be a well formated HTTP headers part including newlines (DOS style) which
# can be entered as escape sequences (\r\n).
# RequestAppendix: X-Tracking-Choice: do-not-track\r\n
# Specifies the IP protocol families to use for remote connections. Order does
# matter, first specified are considered first. Possible combinations:
# v6 v4
# v4 v6
# v6
# v4
# (empty or not set: use system default)
#
# ConnectProto: v6 v4
# Regular expiration algorithm finds package files which are no longer listed
# in any index file and removes them of them after a safety period.
# This option allows to keep more versions of a package in the cache after
# safety period is over.
# KeepExtraVersions: 1
# Optionally uses TCP access control provided by libwrap, see hosts_access(5)
# for details. Daemon name is apt-cacher-ng. Default if not set: decided on
# startup by looking for explicit mentioning of apt-cacher-ng in
# /etc/hosts.allow or /etc/hosts.deny files.
# UseWrap: 0
# If many machines from the same local network attempt to update index files
# (apt-get update) at nearly the same time, the known state of these index file
# is temporarily frozen and multiple requests receive the cached response
# without contacting the server. This parameter (in seconds) specifies the
# length of this period before the files are considered outdated.
# Setting it too low transfers more data and increases remote server load,
# setting it too high (more than a couple of minutes) increases the risk of
# delivering inconsistent responses to the clients.
# FreshIndexMaxAge: 27
# Usually the users are not allowed to specify custom TCP ports of remote
# mirrors in the requests, only the default HTTP port can be used (instead,
# proxy administrator can create Remap- rules with custom ports). This
# restriction can be disabled by specifying a list of allowed ports or 0 for
# any port.
#
# AllowUserPorts: 80
# Normally the HTTP redirection responses are forwarded to the original caller
# (i.e. APT) which starts a new download attempt from the new URL. This
# solution is ok for client configurations with proxy mode but doesn't work
# well with configurations using URL prefixes. To work around this the server
# can restart its own download with another URL. However, this might be used to
# circumvent download source policies by malicious users.
# The RedirMax option specifies how many such redirects the server should
# follow per request, 0 disables the internal redirection. If not set,
# default value is 0 if ForceManaged is used and 5 otherwise.
#
# RedirMax: 5