I think that the spirit of this question, combined with the clarifications in comments, is:

What is it that makes first-order logic "first order"?

Unfortunately, the terms "first order" and "second order" get used to mean various things.

A formal but unsatisfying answer would say that first-order logic is a specific logic defined in, say, Mendelson's textbook, and any other logic is not "first order logic" strictly speaking. This is unsatisfying because we know there are many inessential variations of first-order logic - really there are many first-order *logics* that share a certain core. The question I quoted asks for a characterization of that core.

One common answer is that any logic in which we intend to have quantifiers over "functions" or "sets" is higher order. This is unsatisfying because, as Andrej Bauer points out, such theories can be *syntactically* expressed in multi-sorted first-order logic. There are many theories of "second order arithmetic", for example, which allow us to express set and function quantification but which are treated as first-order theories. Unfortunately, the terminology "second order" is established for these theories and cannot be avoided.

Recall that a logic consists of both a syntax and a semantics. *The truly defining feature of a first-order logic is the semantics.* First-order semantics begins with the notion of a structure (also called a model), as defined in every introductory textbook on first-order logic.

Consider how we would express function quantification in (multi-sorted) first-order logic, as in Andrej's answer. Each structure must interpret two sorts. It uses a set of individuals for the quantifiers over individuals and a separate set of functions for the quantifiers over functions. This set of functions, in an arbitrary structure, might be a proper subset of the collection of *all* functions on the set of individuals; nothing in the definition of a structure requires otherwise. Indeed some structures will have an infinite set of individuals but a finite set of functions.

*Full second order semantics* changes the class of allowable structures so that only those whose function set includes *all* the functions are allowed. This does not affect the syntax in any way, but it deeply changes the semantics. Because fewer structures are being considered, more formulas will be logically valid, and fewer will be satisfiable. Thus there are more categorical theories in these semantics, such as the well known categorical second-order axiomatizations of the natural numbers. Those same axiomatizations are syntactically fine in first-order logic, where the simple difference is that they are no longer categorical.

Thus the key difference between function quantification in multi-sorted first order logic (or type theory) and function quantification in full second-order semantics is not the existence of syntactic quantifier symbols that allow quantification over functions. The difference is in the *meaning* of those quantifiers, which derives from the way the semantics are defined. In the first-order case, we have little control over the range of quantifiers. In full second-order semantics, once the set of individuals is fixed, the range of the function quantifiers is also fixed. This distinction is only visible at the meta level, when we are studying the logic from the outside and can specify which interpretations are permissible. Nothing in the syntax of the logic tells us what collection of structures will be used to interpret it.