unable to split Firebase functions in multiple files

[mInzamamMalik]

I'm working with firebase functions and arrived to hundreds of functions, and now it is very hard to manage it in single index.js file as shown in their lots of examples

I tried to split that functions in multiple files like:

    --firebase.json
    --functions
      --node_modules
      --index.js
      --package.json
      --app
        --groupFunctions.js
        --authFunctions.js
        --storageFunctions.js

in this structure i devide my functions in three categories and put in that three files groupFunctions.js authFunctions.js storageFunctions.js and tried to import that files in index.js but i don't know why it is not working for me

Here is groupFunctions.js

var functions = require('firebase-functions');
module.exports = function(){
    exports.onGroupCreate = functions.database.ref('/groups/{groupId}')
        .onWrite(event => {
            console.log(`A group is created in database named:${event.params.groupId}.`);
            // some logic...
            //...
        })
}

Here is index.js file:

var functions = require('firebase-functions');
module.exports = require("./app/groupFunctions")();

my editor not giving any warning in this code but when i deploy this code with firebase deploy --only functions it does not deploy function (if some functions already exist on firebase console, it remove all functions on deploy)

So I need help regarding this problem,
looking forward to listen from you guise.

question is also asked on stackoverflow

[kibagateaux]

@malikasinger1 I had a similar issue so I had to make a higher order function that takes my firebase objects and returns the cloud function I would like to deploy.

cloudFunction.js

export const cloudFunction = (functions, admin) => {
  return functions.https.onRequest((req, res) => {
    const {userID} = req.body;
    admin.database()
      .ref(`users/${userID}`)
      .update(...storylines)
      .then(snapshot => {
        res.send(snapshot.val());
      });
  });
};

index.js

const functions = require('firebase-functions');
const admin = require('firebase-admin');
const src = require('./src');
admin.initializeApp(functions.config().firebase);
exports.cloudFunctions = src.blah.cloudFunction(functions, admin);

[nicolasgarnier]

Here is my take on it:

You can have an index.js file that will import and list all other Cloud Functions. One trick to improve performance is to use the process.env.FUNCTION_TARGET env variable that will have the name of the Function currently being triggered. During deployment that env variable is null.

Here is an example index.js for 3 functions:

// Sends notifications to users when they get a new follower.
if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === 'sendFollowerNotification') {
  exports.sendFollowerNotification = require('./sendFollowerNotification');
}

// Blur offensive images uploaded on Cloud Storage.
if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === 'blurOffensiveImages') {
  exports.blurOffensiveImages = require('./blurOffensiveImages');
}

// Renders my server side app template.
if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === 'renderTemplate') {
  exports.renderTemplate = require('./renderTemplate');
}

Then for instance, the ./blurOffensiveImages.js would be:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
try {admin.initializeApp(functions.config().firebase);} catch(e) {} // You do that because the admin SDK can only be initialized once.
const mkdirp = require('mkdirp-promise');
// etc... my imports ...

/**
 *  Blur offensive images uploaded on Cloud Storage.
 */
exports = module.exports = functions.storage.object().onChange(event => {
  // Function's code...
});

This will ensure all the 3 functions are deployed if doing a firebase deploy but, at runtime, only one function will be imported. So you won't pollute the functions with each other's imports.

EDIT: Edited to reflect new name of environment variable.

[nicolasgarnier]

Now you could even automate this a bit and register functions based on their file name. For instance with an index.js like this:

**UPDATE, this works:**

const fs = require('fs');
const path = require('path');

// Folder where all your individual Cloud Functions files are located.
const FUNCTIONS_FOLDER = './my_functs';

fs.readdirSync(path.resolve(__dirname, FUNCTIONS_FOLDER)).forEach(file => { // list files in the folder.
  if(file.endsWith('.js')) {
    const fileBaseName = file.slice(0, -3); // Remove the '.js' extension
    if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === fileBaseName) {
      exports[fileBaseName] = require(`${FUNCTIONS_FOLDER}/${fileBaseName}`);
    }
  }
});

Where your file structure is like:

    --firebase.json
    --functions
      --node_modules
      --index.js
      --package.json
      --my_functs
        --sendFollowerNotification.js
        --blurOffensiveImages.js
        --renderTemplate.js

EDIT: Edited to reflect new name of environment variable.

[theprojectsomething]

@nicolasgarnier when attempting to deploy using this method I get the equivalent of
no such file or directory, scandir './my_functs'
assume this is due to the function scanning during deployment (and the fact I'm deploying from the project root, not inside functions). Same goes when attempting to serve from the root.
Is there an obvious way to resolve this? I'm currently checking the dir exists first and if not prepending 'functions/', which works but doesn't seem ideal.

const ENDPOINT_DIR = './endpoints';

fs.readdirSync(fs.existsSync(ENDPOINT_DIR) ? ENDPOINT_DIR : ENDPOINT_DIR.replace('.', './functions')).forEach(file => {
  if(file.endsWith('.js')) {
    const name = file.slice(0, -3);
    const method = name.split(".").reduce(function (method, part) {
      return method ? method + part.charAt(0).toUpperCase() + part.slice(1) : part;
    }, "");
    if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === method) {
      exports[method] = require(`${ENDPOINT_DIR}/${name}`);
    }
  }
});

--firebase.json
    --functions
      --node_modules
      --index.js
      --package.json
      --endpoints
        --system.notify.js

EDIT: Edited to reflect new name of environment variable.

[nicolasgarnier]

Ah yes, I actually had a setup where my index.js was at the root (using "functions": {"source": "./","exclude": "public"}, in my firebase.json where this was working fine. You are right that in the example above we need to prepend 'functions'. I think you can also use __dirname like this:

fs.readdirSync(path.resolve(__dirname, FUNCTIONS_FOLDER)).forEach(file => { ...

If you could try that let me know if that works and I'll update my code above :)

[theprojectsomething]

Works perfectly, thanks.

[oodavid]

I've created a slightly modified version of index.js that uses globbing to search for files ending with .function.js. This allows me to organise my directory in any manner.

/** EXPORT ALL FUNCTIONS
 *
 *   Loads all `.function.js` files
 *   Exports a cloud function matching the file name
 * 
 *   Based on this thread:
 *     https://github.com/firebase/functions-samples/issues/170
 */
const glob = require("glob");
const files = glob.sync('./**/*.function.js', { cwd: __dirname });
for(let f=0,fl=files.length; f<fl; f++){
  const file = files[f];
  const functionName = file.split('/').pop().slice(0, -12); // Strip off '.function.js'
  if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === functionName) {
    exports[functionName] = require(file);
  }
}

EDIT: Edited to reflect new name of environment variable.

[sonovice]

Thanks for the code snippet, @oodavid . Would it be possible to parse the names of the subfolders as well, e.g. deploy the function in functions/geo/spatial.function.js to https://YOUR_FUNCTIONS_URL/geo/spatial?

[oodavid]

@sonovice you could add a library like camelCase and do something like this to generate your function names, ie:

const functionName = camelCase(file.slice(0, -12).split('/'));

this would set the name of functions/path/to/my/logic.function.js to pathToMyLogic.

(untested logic)

[andrewspy]

@oodavid I have created a simple function at functions/hello/helloWorld.js as follow:-

'use strict';

const functions = require('firebase-functions');

exports = module.exports = functions.https.onRequest((request, response) => {
  response.send('Hello from Firebase!');
});

But it fails with ! functions[helloWorld]: Deploy Error: Function load error: Error: function load attempt timed out.

Did I missed out anything?

[prescottprue]

@andrewspy Do have a top level exports.helloWorld in functions/index that references the export you showed above?

Something like this:

const helloWorld = require('./hello/helloWorld')
module.exports.helloWorld = helloWorld

Checkout @nicolasgarnier 's comment above (notice how it sets exports.someName to the function).

[andrewspy]

@prescottprue I am actually using @oodavid's code from #170 (comment), which set the reference in index.js in the following code:-

  if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === functionName) {
    exports[functionName] = require(file);
  }

I am new to node.js, it'll be great if someone can share a complete working code with index.js and helloWorld.function.js. Thanks!

EDIT: Edited to reflect new name of environment variable.

[oodavid]

@andrewspy with the index.js file I suggested, I'm globbing for files that match './**/*.function.js', so helloWorld.js won't get matched. Rename it to helloWorld.function.js and see if that changes anything.

If not, I'll make an example repo.

[andrewspy]

@oodavid I am actually using functions/hello/helloWorld.function.js, and it's not working... below is the console output for firebase deploy --only functions

i  deploying functions
i  functions: ensuring necessary APIs are enabled...
i  runtimeconfig: ensuring necessary APIs are enabled...
+  runtimeconfig: all necessary APIs are enabled
+  functions: all necessary APIs are enabled
i  functions: preparing functions directory for uploading...
i  functions: packaged functions (3.88 KB) for uploading
+  functions: functions folder uploaded successfully
i  starting release process (may take several minutes)...
i  functions: updating function helloWorld...
!  functions[helloWorld]: Deploy Error: Function load error: Error: function load attempt timed out.

[edTheGuy00]

after trying most of these solutions, only this one worked for me

index.js

'use strict';

//Declare all your child functions here
const fooFunction = require('./foo');
const barFunction = require('./bar');

// Note: these tasks need to be initialized in index.js and
// NOT in child functions:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const database = admin.database();
const messaging = admin.messaging();

exports.fooFunction = functions.database.ref('/users/messages_inbox/{sender_id}/message').onWrite(event => {
// Pass whatever tasks to child functions so they have access to it
  fooFunction.handler(database, messaging, event);
});

exports.barFunction = functions.database.ref('/users').onWrite(event => {
// Pass whatever tasks to child functions so they have access to it
  barFunction.handler(database);
});

foo.js

exports.handler = function(database, messaging, event) {
       // some function
}

bar.js

exports.handler = function(database) {
       // some function
}

[TarikHuber]

@oodavid thx for the great example! 😄
Maybe a little modification that could help. This should remove the problem for @andrewspy

/** EXPORT ALL FUNCTIONS
 *
 *   Loads all `.function.js` files
 *   Exports a cloud function matching the file name
 *
 *   Based on this thread:
 *     https://github.com/firebase/functions-samples/issues/170
 */
const glob = require("glob");
const files = glob.sync('./**/*.function.js', { cwd: __dirname, ignore: './node_modules/**'});
for(let f=0,fl=files.length; f<fl; f++){
  const file = files[f];
  const functionName = file.split('/').pop().slice(0, -12); // Strip off '.function.js'
  if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === functionName) {
    exports[functionName] = require(file);
  }
}

If you don't ignore the node_modules and you have a lot of them the functions deployment could time out. It will deploy all functions but we don't need it to search over the node_modules folder.

And just for the record if someone gets an error for the firebase-admin initialization. The way @nicolasgarnier wrote his functions is the one that should work.

const functions = require('firebase-functions');
const admin = require('firebase-admin');
try {admin.initializeApp(functions.config().firebase);} catch(e) {} // You do that because the admin SDK can only be initialized once.
const mkdirp = require('mkdirp-promise');
// etc... my imports ...

/**
 *  Blur offensive images uploaded on Cloud Storage.
 */
exports = module.exports = functions.storage.object().onChange(event => {
  // Function's code...
});

The try catch does the trick 😉

And the camelCase worksout with this little change:

const functionName = camelCase(file.slice(0, -12).split('/').join('_'));

EDIT: Edited to reflect new name of environment variable.

[mdrideout]

I edited this script to allow for multiple functions within each document

Where each file may export multiple functions

const functions = require('firebase-functions');

const query = functions.https.onRequest((req, res) => {
    let query = req.query.q;

    res.send({
        "You Searched For": query
    });
});

const searchTest = functions.https.onRequest((req, res) => {
    res.send({
        "searchTest": "Hi There!"
    });
});

module.exports = {
    query,
    searchTest
}

And the script goes through each one identified in each file

// Folder where all your individual Cloud Functions files are located.
const FUNCTIONS_FOLDER = './scFunctions';

fs.readdirSync(path.resolve(__dirname, FUNCTIONS_FOLDER)).forEach(file => { // list files in the folder.
  if(file.endsWith('.js')) {
    const fileBaseName = file.slice(0, -3); // Remove the '.js' extension
    const thisFunction = require(`${FUNCTIONS_FOLDER}/${fileBaseName}`);
    for(var i in thisFunction) {
        exports[i] = thisFunction[i];
    }
  }
});

The URL's on firebase are predictably named

✔ functions: query: http://localhost:5001/PROJECT-NAME/us-central1/query
✔ functions: helloWorlds: http://localhost:5001/PROJECT-NAME/us-central1/helloWorlds
✔ functions: searchTest: http://localhost:5001/PROJECT-NAME/us-central1/searchTest

[finestdesign]

Hi, Please help. I want to write a program on our own https server to add two number without use fire base.
So please to run on google home .

[armenr]

This thread is great. Props to @TarikHuber for his Medium post that brought me here.

My working implementation for single function/file is as follows:

const glob = require("glob");
const camelCase = require("camelcase");
const files = glob.sync("./**/*.f.js", {
  cwd: __dirname,
  ignore: "./node_modules/**"
});

files.forEach(functionFile => {
  let functionName = camelCase(
    functionFile
      .slice(0, -5)
      .split("/")
      .join("_")
  );

  !process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === functionName
    ? (exports[functionName] = require(functionFile))
    : console.log('Something went wrong!');
});

EDIT: Edited to reflect new name of environment variable.

[alixaxel]

The FUNCTION_TARGET trick could be written in a more concise way:

if ((process.env.FUNCTION_TARGET || functionName) === functionName) {
  exports[functionName] = require(functionName);
}

EDIT: Edited to reflect new name of environment variable.

[troy21688]

I am having an issue similar to the above, please see by SO post: https://stackoverflow.com/questions/50089807/firebase-cloud-functions-functions-predeploy-error-when-structuring-multiple-f

[marioavs]

Following @boon4376 previous comment, I prefer multiple functions in each file. I have added some "common" files like the following to avoid the SDK being initialized multiple times:

admin.js

// The Firebase Admin SDK to access the Firebase Realtime Database.
const admin = require('firebase-admin');
admin.initializeApp();

module.exports = admin;

The functions are implemented as follows.

realtimedb.js

const functions = require('firebase-functions');
const admin = require('./admin');
// ... more imports

exports.onDeleteItemA = functions.database.ref('/ItemA/{itemId}')
  .onDelete((snapshot, context) => {
// code
// admin.database().ref('/somePath')...
  });

exports.onDeleteItemB = functions.database.ref('/ItemB/{itemId}')
  .onDelete((snapshot, context) => {
// code
  });

And the main index.js file is just excluding some particular files.

index.js

const fs = require('fs');
const path = require('path');

// Folder where all your individual Cloud Functions files are located.
const FUNCTIONS_FOLDER = './';

fs.readdirSync(path.resolve(__dirname, FUNCTIONS_FOLDER))
  .forEach(file => { // list files in the folder.
    if ((file.endsWith('.js')) && (file !== 'index.js') && (file !== 'admin.js')) {
      const fileBaseName = file.slice(0, -3); // Remove the '.js' extension
      const exportedModule = require(`${FUNCTIONS_FOLDER}/${fileBaseName}`);
      for(var functionName in exportedModule) {
        if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === functionName) {
          exports[functionName] = exportedModule[functionName];
        }
      }
    }
  });

EDIT: Edited to reflect new name of environment variable.

[yuliankarapetkov]

@nicolasgarnier thanks for sharing the process.env.FUNCTION_TARGET trick.

However, does this mean that we practically can't make use of express routing if we want to boost performance? The FUNCTION_TARGET variable of an API will always be one and the same (in the example below api), whichever route you are making a request to.

const express = require('express');
const app = express();

app.get("/users/:uid/posts", async (req, res) => {
   // process.env.FUNCTION_TARGET === 'api'
});

app.post("/users/:uid/posts", async (req, res) => {
   // process.env.FUNCTION_TARGET === 'api'
});

exports.api = functions.https.onRequest(app);

EDIT: Edited to reflect new name of environment variable.

[nicolasgarnier]

Well this trick is specifically made when using different Cloud Functions. In your case everything is in the same cloud function but you can still "lazy-load" dependencies the normal way using require() if you want. Like this for instance:

const express = require('express');
const app = express();

app.get("/users/:uid/posts", async (req, res) => {
   require("./users_get.js").default(req, res);
});

app.post("/users/:uid/posts", async (req, res) => {
   require("./users_post.js").default(req, res);
});

exports.api = functions.https.onRequest(app);

And you'd have the two files "./users_get.js" and "./users_post.js" doing whatever is only needed for one specific handler.

This optimisation is not that useful though because eventually both handlers run on the same Cloud functions instance so they will get both called on the same instance after a while. But still you could do it, it would help a little bit spread the static init over time.

In the case of entirely different Cloud Functions we do this optimisation because we're entirely sure that the code will never be ran on the same instance so there is really no point in doing all the static initialisation for other functions.

[yuliankarapetkov]

Makes sense, thanks!

[elis]

I've been researching about scalable and convenient firebase functions project structure, and came up with this thread. My implementation below allows for standard firebase function naming convention for objects that will follow your folder structure.

import { merge } from 'lodash'
import { sync } from 'glob'

const functionsRoot = 'root'

const files = sync(`./${functionsRoot}/**/*.f.js`, { cwd: __dirname, ignore: './node_modules/**'});

const nameIt = (name, file) => {
  const qname = name.slice(0, -5).split('/').filter(e => e) // Strip off '.f.js'
  const first = qname.shift()
  const fname = [first, ...qname].join('-')
  if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === fname) {
    // lazy loading - require file only when building or running function
    const exported = require(file)
    const subject = qname.reverse().reduce((obj, el) => ({[el]: obj}), exported.default || exported)
    exports[first] = qname.length ? merge({}, exports[first] || {}, subject) : subject
  }
}

for (let f=0,fl=files.length; f<fl; f++) {
  const file = files[f];
  const name = file.replace(new RegExp(`^./${functionsRoot}`), '')
  nameIt(name, file)
}

This will allow you to have a folder structure like:

root
  users
    initUser.f.js
    findUsers.f.js
    admin
      deleteUser.f.js
  upgradePost.f.js

and an export structure like:

exports = {
  users: {
    initUser: fn(),
    findUsers: fn(),
    admin: {
      deleteUser: fn()
    }
  },
  upgradePost: fn()
}

One thing I didn't like with the original suggestion by @armenr is that it collapsed the function name into a single string and bypassed an entire feature that allows structuring the functions based on objects exported.

This not only a styling issue making function names really long and basically hard to use, but also a functional issue as it disables your ability to deploy a group of functions using something like firebase deploy --only function:users.admin

EDIT: Edited to reflect new name of environment variable.

[chadRoberge]

similar to @nicolasgarnier above I would like to create a file structure for api like so:

--functions
----api
-------v1
----------deposit.api.js
----------invoice.api.js
-------v2
----------deposit.api.js

I would like to use @TarikHuber version in index.js already have it separating the file types like so:

const glob = require("glob");
const camelCase = require("camelcase");
const files = glob.sync('./**/*.js', { cwd: __dirname, ignore: ['./node_modules/**', './index.js'] });
for (let f = 0, fl = files.length; f < fl; f++) {
  const file = files[f];
  const pathCheck = file.slice(-6, -3);
  if (pathCheck === 'api') {
    const path = file.slice(0, -7); // Strip off '.api.js'
    app.use(`/${path}`, require(file));    
  } else {
    const functionName = camelCase(file.slice(0, -5).split('/').join('_')); // Strip off '.f.js'
    if (!process.env.FUNCTION_TARGET || process.env.FUNCTION_TARGET === functionName) {
      exports[functionName] = require(file);
    }
  }
}

new to node so not exactly sure why this shouldn't work.

Thanks for any help in advance...

Chad

EDIT: Edited to reflect new name of environment variable.

[theprojectsomething]

I've had a bullet-proof version of the original approach from @nicolasgarnier running for a few years now. No problems other than a want to revisit to allow globbing inside a deeply nested folder structure (not least to make use of selective group deployments).

Having finally had the chance to implement on a new project I quickly discovered the require('./../../../../') woes of deep nesting. Fortunately I was already using dot.notation to flag non-deploying and non-serving files, and so used this technique to allow grouping as well.

See the gist below for my updated multi-file function setup. It's tested and working for the latest version of FB Cloud Functions with the following features:

• deploy all files under a designated directory as cloud functions
• use dot notation to create function groups
• alternatively allows deep nesting to group functions as well
• allows path flags to ignore files (i.e. don't create a function with them) or only serve them locally (to allow high level admin or debugging functionality that won't be deployed)

https://gist.github.com/theprojectsomething/2076f856f9c4488366dc88e6e8ab2f20

[george43g]

Hey everyone! After doing a bit of research and finding this thread among others, I've decided to release this solution as a package, better-firebase-functions, which takes into account the majority of use-cases outlined by everyone here.

Your index.js file needs only 2 lines

import exportCloudFunctions from 'better-firebase-functions'
exportCloudFunctions(__dirname, __filename, exports, './', GLOB_PATTERN);

https://www.npmjs.com/package/better-firebase-functions
https://github.com/gramstr/better-firebase-functions

This is my first open-source repo, so let me know what you guys think. Any pull requests welcome!

• any file / dir structure you want, dots and dashes in filenames and folder names auto-handled
• dots, dashes turned into camelCase
• any number of file extension chars or segments, automatically handled
• every file exports one default func
• correctly nested exports object
• lightweight, 3 deps
• we can all contribute and improve module to make it more robust & resistant to edge cases, such as auto-detecting conflicts and incorrect exports

[truefedex]

With typescript >=3.8 just write in index.ts something like:

export * from './functions/triggers';
export * from './functions/endpoints';
...

https://www.typescriptlang.org/docs/handbook/modules.html#export-all-as-x

[manwithsteelnerves]

Anyone can give an idea on how to use the technique when using functions with express framework?
At the top index.ts file I will be exporting the functions. And almost all files has some form of import statements. How can I control so that for each function I deploy, whole source code is not pushed to the firebase server?
Yes, as of now, if there are 10 different functions in a project, all 10 functions has same code (complete project code in each function!!!) deployed to the firebase server.

index.ts

// Init express
const app = express();

// Allow cross-origin requests middleware
app.use(cors({ origin: true }));
app.use(morgan('dev'));
app.use("/", versions.router);
  
// Fallback for rest
app.use("*", fallback);

app.use(errorHandlerCallback);

// Redirect all to each domain function and let express routes handle
const environmentFunction = (expressApp : express.Express, environment : Environment) => {
  return functions.https.onRequest((req : express.Request, res : express.Response) => {
    req.environment = environment;
    return expressApp(req, res);
  });
};

exports.admin = environmentFunction(app, 'admin');
exports.b2b   = environmentFunction(app, 'b2b');
exports.b2c   = environmentFunction(app, 'b2c');
exports.sdk   = environmentFunction(app, 'sdk');

// Export subscribers and triggers
exports.subscribers = subscribers;
exports.triggers    = triggers;
exports.webhooks    = webhooks;

[jdgamble555]

Simple way to chose the functions you want:

const FUNC = process.env.FUNCTION_TARGET;
const cols = ['posts', 'users'];

cols.forEach(col => {
    if (!FUNC || FUNC === col) {
        exports[col] = require('./' + col);
    }
});

EDIT: Edited to reflect new name of environment variable.

[katowulf]

Now a documented feature: https://firebase.google.com/docs/functions/organize-functions#write_functions_in_multiple_files

[schmidt-sebastian]

Please note that the name of the environment variable has been changed from FUNCTION_NAME to FUNCTION_TARGET.