You know the game:
tick-tack-toe  naughts-and-crosses  Xs&Os
Let’s ask some questions:
Q: Is it a two player game?
A: yes
Q: Is is a game of chance?
A: no
Q: Is it a fair game?
A: yes (intuitively proving that to ourselves is a lot of the fun of the game!)
Q: Is the first player’s favorite color red?
We know that question is not like the others right away. We cannot answer that question only on the information in the rules. That question is contingent on something other than the rules!

Mathematics can be complicated enough that we don’t know right away that a question is missing information. That missing information may be hidden in definitions. Here are four examples of definitions that are subtlety different than how they are often understood and taught. 

1.
natural numbers as sets
An early lesson in set theory is to define natural numbers in terms of sets. One way to define natural numbers is: n + 1 = s(n) = {n} U n. This way is common because proofs are easy. Some teachers show an alternative definition of natural numbers: n + 1 = s(n) = {n}. It is ironic that set theory has its place in the curriculum to teach the foundations of mathematics and that with our first lesson we gloss over the fact that we stipulate definitions. If we do not stipulate these definitions (and in so doing implicitly add axioms!), then the ‘theorems’ are not theorems at all. Since potential definitions are not unique, they are not valid without adding axioms. (We can’t just add clauses to our conjectures, instead of adding axioms, because definitions of numbers require axiom schema.)

2.
real numbers as continuous
Notice that Dedeakind cuts are less popular definition than Cauchy sequences. Students and teachers are discomforted by the question, “How do we make the cut?” While we may be reassured with examples of sequences that are Cauchy, we should not overlook that we have the same issue. All our example sequences are generated; generated sequences, all together, are countable. To select sequences that are Cauchy, we may use the axiom of choice. The axiom of choice is non-constructive, so that the Cauchy sequences may be uncountable. The operative phrase here is ‘may be’. ZFC does not have a combination of axioms that makes the real numbers (defined via Cauchy sequences) continuous.

3.
computation vs computeability
We are so accustomed to interactive programs that might forget that computation only produces output when (after) it halts. We cannot just map an algorithm over natural numbers. We cannot extend computation to infinity because the theory of computation is a way of expressing every permutation and combination. (consistency)

4.
undefineability of the unorderedness of sets
The first thing to know about sets is that they are unordered collections. We must have sets before we can have set theory. The unorderedness of sets cannot be directly defined in set theory. (see AC / Zorn’s lemma / well ordering vs stratified set theory)

We can see that we should always prefer fully decideable definitions for the foundations of mathematics. When we use definitions that are not fully decideable, then they depend on some information that is not in the rules. For such definitions, each statement can only be a theorem with the inclusion of the definition in the statement. That is very inconvenient and unsatisfactory. 

Deductions are part of deductive reasoning, but what cannot be deduced cannot be part of deductive reasoning. Semi-decideable concepts / meta-mathematics forms a boundary of mathematics. A notable example is mathematical induction. 

When we want mathematical induction to go beyond a finite amount, we use the axiom of infinity. Let us use the completeness of first order logic to consider what the syntactic equivalent of the axiom of infinity is. The way that the axiom of infinity works is that the infinity is generated, so the syntactic expression of the generator is a formula. Adding the axiom of infinity to the other axioms, effectively, makes what was a first order theory into a second order theory. That is why it is incomplete (see Godel's incompleteness theorems). 

To summarize, we critique the teaching of the foundations of mathematics. Defineability is in correspondence with decideability. Concepts that are semi-decideable for a deductive system form an open boundary of that system. The axiom of infinity corresponds to the difference between first order logic and second order logic. Studying the foundations of mathematics we should not confuse possibility with necessity.