This also includes a refactor of the hugofs package and its usage.
The motivation for that is:
The Afero filesystems are brilliant. Hugo's way of adding a dozen of global variables for the different filesystems was a mistake. In readFile (and also in some other places in Hugo today) we need a way to restrict the access inside the working dir. We could use ioutil.ReadFile and implement the path checking, checking the base path and the dots ("..") etc. But it is obviously better to use an Afero BasePathFs combined witha ReadOnlyFs. We could create a use-once-filesystem and handle the initialization ourselves, but since this is also useful to others and the initialization depends on some other global state (which would mean to create a new file system on every invocation), we might as well do it properly and encapsulate the predefined set of filesystems. This change also leads the way, if needed, to encapsulate the file systems in a struct, making it possible to have several file system sets in action at once (parallel multilanguage site building? With Moore's law and all...)
Fixes#1551
Simple ioutil.ReadFile is used for reading file contents but it reads
all of the file contents and copies them into the memory and is run in a
single goroutine. It causes much memory consumption at copying media
files in content directory to publish directory and it is not good at
performance.
This improves the both issue by replacing ReadFile with LazyFileReader.
It postpones reading the file contents until it is really needed. As the
result, actual file read is run in parallelized goroutine. It improves
performance especially in a really big site.
In addition, if this reader is called from io.Copy, it does not copy the
file contents into the memory but just copies them into destination
file. It improves much memory consumption issue when the site has many
media files.
Fix#1181
LazyFileReader is an io.Reader implementation to postpone reading the
file contents until it is really needed. It is introduced for improving
performance and memory consumption at reading media files in content
directory.
The following inside `config.toml` will ignore files ending with `.foo` and `.boo`.
```
watchIgnoreFiles = [ "\\.foo$", "\\.boo$" ]
```
The above is is a list of Reqular Expressions, but note the escaping of the `\` to make TOML happy.
Fixes#1189
First step to use initialisms that golint suggests,
for example:
Line 116: func GetHtmlRenderer should be GetHTMLRenderer
as see on http://goreportcard.com/report/spf13/hugo
Thanks to @bep for the idea!
Note that command-line flags (cobra and pflag)
as well as struct fields like .BaseUrl and .Url
that are used in Go HTML templates need more work
to maintain backward-compatibility, and thus
are NOT yet dealt with in this commit.
First step in fixing #959.
filepath.Walk does not follow symbolic links.
There's no easy fix for that outside of Go, so the best we can do for now is to give notice to the end user by ERROR log statements.
This commit also fixes a related panic situation in GenerateTemplateNameFrom when the layout dir was a symbolic link.
Fixes#283
File handling was broken on Windows. This commit contains a revision of the path handling with separation of file paths and urls where needed.
There may be remaining issues and there may be better ways to do this, but it is easier to start that refactoring job with a set of passing tests.
Fixes#687Fixes#660
The front matter convert-feature didn't work. It placed converted content in the wrong directory.
This commit fixes this by doing the smallest and safest change possible; the path logic here should maybe
be revisited and generalized.
Fixes#643
filepath was used inconsistently throughout the hugolib. With the
introduction of source and target modules, all path are normalized to
"/". This simplifies the processing of paths. It does mean that
contributors need to be aware of using path/filepath in any module other
than source or target is not recommended. The current exception is
hugolib/config.go
