Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Safe default filename #689

Merged
merged 34 commits into from
Mar 18, 2021
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
777a04f
feat: allow to use filename option without keepExtension
GrosSacASac Feb 18, 2021
5531a57
feat: can use uploadDir option and filename together
GrosSacASac Feb 18, 2021
33dd6c4
refactor: rename
GrosSacASac Feb 18, 2021
ebb7a18
feat: by default prevent directory traversal attacks
GrosSacASac Feb 18, 2021
cb3d359
refactor: make octet stream less divergent
GrosSacASac Feb 18, 2021
aafebea
refactor: pass through, avoid renaming variable names
GrosSacASac Feb 18, 2021
0a5bad4
refactor: prefer Object.assign
GrosSacASac Feb 18, 2021
032e66b
feat: pass newName
GrosSacASac Feb 18, 2021
ad1664a
feat: give createFileWriteStream more, including newName
GrosSacASac Feb 18, 2021
c178080
docs: update
GrosSacASac Feb 18, 2021
f600f32
docs: update examples
GrosSacASac Feb 18, 2021
3ca6382
fix: fix missing variables
GrosSacASac Feb 18, 2021
df66b4f
feat: remove duplicate fix tests
GrosSacASac Feb 18, 2021
e0576e1
lint: lint
GrosSacASac Feb 18, 2021
59a9b2b
refactor: rename mime into mimetype
GrosSacASac Feb 19, 2021
2ca0b78
refactor: explicit this.options.keepExtensions !== true
GrosSacASac Feb 19, 2021
e4b7fe6
tests: reverse expectation order
GrosSacASac Feb 19, 2021
3481057
tests: rename to xname to avoid confusion
GrosSacASac Feb 19, 2021
bef35f2
refactor: inline old _uploadPath
GrosSacASac Feb 19, 2021
45c9213
refactor: rename newName into newFileName
GrosSacASac Feb 19, 2021
7edf4bb
refactor: rename filename into originalFilename
GrosSacASac Feb 19, 2021
8340df2
refactor: direcly filepath
GrosSacASac Feb 26, 2021
63402a1
fix: test
GrosSacASac Feb 26, 2021
9e54b19
refactor: split hash into hashAlgorithm and hash
GrosSacASac Feb 26, 2021
4f09883
feat: this.lastModifiedDate = null remains
GrosSacASac Feb 26, 2021
2d85616
refactor: finalpath: filepath
GrosSacASac Feb 26, 2021
48f25fc
fix: change order
GrosSacASac Feb 26, 2021
3c14493
refactor: better be explicit
GrosSacASac Feb 26, 2021
58e4a61
feat: display more in toString
GrosSacASac Feb 26, 2021
4ff8447
docs: update changelog
GrosSacASac Mar 2, 2021
fec82d3
chore: update version
GrosSacASac Mar 2, 2021
738ff5c
Merge branch 'master' into safe-default-filename
GrosSacASac Mar 3, 2021
4bce5f7
fix: revert, renamed too much
GrosSacASac Mar 4, 2021
97181fe
fix: _flush is more appropriate
GrosSacASac Mar 5, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,10 @@ See it's defaults in [src/Formidable.js DEFAULT_OPTIONS](./src/Formidable.js)
files for inputs which submit multiple files using the HTML5 `multiple`
attribute. Also, the `fields` argument will contain arrays of values for
fields that have names ending with '[]'.
- `options.filename` **{function}** - default `undefined` Use it to control
newName. Must return a string. Will be joined with options.uploadDir.

#### `options.filename` **{function}** function (name, ext, part, form) -> string

_**Note:** If this size of combined fields, or size of some file is exceeded, an
`'error'` event is fired._
Expand Down Expand Up @@ -567,7 +571,7 @@ form.onPart = function (part) {
// let formidable handle only non-file parts
if (part.filename === '' || !part.mime) {
// used internally, please do not override!
form.handlePart(part);
form._handlePart(part);
}
};
```
Expand Down Expand Up @@ -634,8 +638,9 @@ file system.
```js
form.on('fileBegin', (formName, file) => {
// accessible here
// formName the name in the form (<input name="thisname" type="file">)
// formName the name in the form (<input name="thisname" type="file">) or http filename for octetstream
// file.name http filename or null if there was a parsing error
// file.newName generated hexoid or what options.filename returned
// file.path default pathnme as per options.uploadDir and options.filename
// file.path = CUSTOM_PATH // to change the final path
});
Expand Down
2 changes: 1 addition & 1 deletion examples/log-file-content-to-console.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ const server = http.createServer((req, res) => {
if (req.url === '/api/upload' && req.method.toLowerCase() === 'post') {
// parse a file upload
const form = formidable({
fileWriteStreamHandler: () => {
fileWriteStreamHandler: (/* file */) => {
const writable = Writable();
// eslint-disable-next-line no-underscore-dangle
writable._write = (chunk, enc, next) => {
Expand Down
4 changes: 2 additions & 2 deletions examples/store-files-on-s3.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ const s3Client = new AWS.S3({
},
});

const uploadStream = (filename) => {
const uploadStream = (file) => {
const pass = PassThrough();
s3Client.upload(
{
Bucket: 'demo-bucket',
Key: filename,
Key: file.newName,
Body: pass,
},
(err, data) => {
Expand Down
23 changes: 15 additions & 8 deletions examples/with-http.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,22 @@ const formidable = require('../src/index');
const server = http.createServer((req, res) => {
if (req.url === '/api/upload' && req.method.toLowerCase() === 'post') {
// parse a file upload
const form = formidable({ multiples: true });

form.on('fileBegin', (formName, file) => {
if (file.name === null) {
// todo change lint rules because that is what we recommend
// eslint-disable-next-line
file.name = 'invalid-characters';
}
const form = formidable({
multiples: true,
uploadDir: `uploads`,
keepExtensions: true,
filename(/*name, ext, part, form*/) {
/* name basename of the http filename
ext with the dot ".txt" only if keepExtension is true
*/
// slugify to avoid invalid filenames
// substr to define a maximum length
// return `${slugify(name).${slugify(ext, separator: '')}`.substr(0, 100);
return 'yo.txt'; // or completly different name
// return 'z/yo.txt'; // subdirectory
},
});

form.parse(req, (err, fields, files) => {
if (err) {
console.error(err);
Expand Down
58 changes: 42 additions & 16 deletions src/Formidable.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ const DEFAULT_OPTIONS = {
multiples: false,
enabledPlugins: ['octetstream', 'querystring', 'multipart', 'json'],
fileWriteStreamHandler: null,
defaultInvalidName: 'invalid-name',
};

const PersistentFile = require('./PersistentFile');
Expand All @@ -47,7 +48,9 @@ class IncomingForm extends EventEmitter {

this.options = { ...DEFAULT_OPTIONS, ...options };

const dir = this.options.uploadDir || this.options.uploaddir || os.tmpdir();
const dir = path.resolve(
this.options.uploadDir || this.options.uploaddir || os.tmpdir(),
);

this.uploaddir = dir;
this.uploadDir = dir;
Expand Down Expand Up @@ -316,8 +319,11 @@ class IncomingForm extends EventEmitter {

this._flushing += 1;

const newName = this._getNewName(part);
const finalPath = this._joinDirectoryName(newName);
const file = this._newFile({
path: this._rename(part),
newName,
path: finalPath,
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
filename: part.filename,
mime: part.mime,
});
Expand Down Expand Up @@ -477,18 +483,21 @@ class IncomingForm extends EventEmitter {
return new MultipartParser(this.options);
}

_newFile({ path: filePath, filename: name, mime: type }) {
_newFile({ path: filePath, filename, mime, newName }) {
return this.options.fileWriteStreamHandler
? new VolatileFile({
name,
type,
newName,
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
path: filePath, // avoid shadow
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
filename,
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
mime,
createFileWriteStream: this.options.fileWriteStreamHandler,
hash: this.options.hash,
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
})
: new PersistentFile({
newName,
path: filePath,
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
name,
type,
filename,
mime,
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
hash: this.options.hash,
});
}
Expand Down Expand Up @@ -527,8 +536,8 @@ class IncomingForm extends EventEmitter {
return basename.slice(firstDot, lastDot) + extname;
}

_uploadPath(part, fp) {
const name = fp || `${this.uploadDir}${path.sep}${toHexoId()}`;
_uploadPath(part) {
const name = toHexoId();

if (part && this.options.keepExtensions) {
const filename = typeof part === 'string' ? part : part.filename;
Expand All @@ -538,17 +547,34 @@ class IncomingForm extends EventEmitter {
return name;
}

_setUpRename() {
const hasRename = typeof this.options.filename === 'function';
_joinDirectoryName(name) {
const newPath = path.join(this.uploadDir, name);

// prevent directory traversal attacks
if (!newPath.startsWith(this.uploadDir)) {
return path.join(this.uploadDir, this.options.defaultInvalidName);
}

if (this.options.keepExtensions === true && hasRename) {
this._rename = (part) => {
const resultFilepath = this.options.filename.call(this, part, this);
return newPath;
}

return this._uploadPath(part, resultFilepath);
_setUpRename() {
const hasRename = typeof this.options.filename === 'function';
if (hasRename) {
this._getNewName = (part) => {
let ext = '';
let name = this.options.defaultInvalidName;
if (part.filename) {
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
// can be null
({ ext, name } = path.parse(part.filename));
if (!this.options.keepExtensions) {
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
ext = '';
}
}
return this.options.filename.call(this, name, ext, part, this);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

path's return of ext isn't handling well filenames with multiple dots, that's why i did the _getExtension method, maybe use it here too.

this._getNewName = (part) => {
  let filename = this.options.defaultInvalidName
  let extname = '';

  if (part.mimetype) {
    filename = part.originalFilename;
    if (this.options.keepExtensions === true) {
      extname = this._getExtename(part.originalFilename);
    }
  }

  return this.options.filename.call(this, filename, extname, part, this);
};

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but the problem is that there would be duplicated parts. For example x.y.z.w would call
this.options.filename.call(this, name, ext, part, this);
with name = 'x.y.z' and ext = '.y.z.w'

};
} else {
this._rename = (part) => this._uploadPath(part);
this._getNewName = (part) => this._uploadPath(part);
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand Down
1 change: 0 additions & 1 deletion src/FormidableError.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
/* eslint-disable no-plusplus */


const missingPlugin = 1000;
const pluginFunction = 1001;
const aborted = 1002;
Expand Down
20 changes: 5 additions & 15 deletions src/PersistentFile.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,11 @@ class PersistentFile extends EventEmitter {
constructor(properties) {
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
super();

this.size = 0;
this.path = null;
this.name = null;
this.type = null;
this.hash = null;
this.lastModifiedDate = null;
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
Object.assign(this, properties);
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved

this.size = 0;
this._writeStream = null;

// eslint-disable-next-line guard-for-in, no-restricted-syntax
for (const key in properties) {
this[key] = properties[key];
}

if (typeof this.hash === 'string') {
this.hash = crypto.createHash(properties.hash);
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
} else {
Expand All @@ -42,12 +33,11 @@ class PersistentFile extends EventEmitter {
const json = {
Copy link
Member

@tunnckoCore tunnckoCore Feb 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use vfile?

const json = VFile({
  path: this.filepath,
  mtime: this.lastModifiedDate,
  originalName: this.originalFilename,
  mimetype: this.mimetype,
  size: this.size,
  hash: this.hashAlgorithm && this.hashAlgorithm !== '' ? this.hash : '',
});

same for VolatileFile

using vfile gives us cool unification and naming, or at least naming that most are used to and know what is what... extname, dirname, basename, stem (basename without ext)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, this should be in the constructor

this._file = VFile(...)

then in toJSON just return this._file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this PR is already too big, maybe do it in another ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep.

size: this.size,
path: this.path,
name: this.name,
type: this.type,
newName: this.newName,
mime: this.mime,
mtime: this.lastModifiedDate,
length: this.length,
filename: this.filename,
mime: this.mime,
};
if (this.hash && this.hash !== '') {
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
json.hash = this.hash;
Expand All @@ -56,7 +46,7 @@ class PersistentFile extends EventEmitter {
}

toString() {
return `PersistentFile: ${this.name}, Path: ${this.path}`;
return `PersistentFile: ${this.filename}, Path: ${this.path}`;
GrosSacASac marked this conversation as resolved.
Show resolved Hide resolved
}

write(buffer, cb) {
Expand Down
18 changes: 5 additions & 13 deletions src/VolatileFile.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,11 @@ class VolatileFile extends EventEmitter {
constructor(properties) {
super();

this.size = 0;
this.name = null;
this.type = null;
this.hash = null;
Object.assign(this, properties);

this.size = 0;
this._writeStream = null;

// eslint-disable-next-line guard-for-in, no-restricted-syntax
for (const key in properties) {
this[key] = properties[key];
}

if (typeof this.hash === 'string') {
this.hash = crypto.createHash(properties.hash);
} else {
Expand All @@ -29,7 +22,7 @@ class VolatileFile extends EventEmitter {
}

open() {
this._writeStream = this.createFileWriteStream(this.name);
this._writeStream = this.createFileWriteStream(this);
this._writeStream.on('error', (err) => {
this.emit('error', err);
});
Expand All @@ -42,8 +35,7 @@ class VolatileFile extends EventEmitter {
toJSON() {
const json = {
size: this.size,
name: this.name,
type: this.type,
newName: this.newName,
length: this.length,
filename: this.filename,
mime: this.mime,
Expand All @@ -55,7 +47,7 @@ class VolatileFile extends EventEmitter {
}

toString() {
return `VolatileFile: ${this.name}`;
return `VolatileFile: ${this.filename}`;
}

write(buffer, cb) {
Expand Down
Loading