python - Best way to replace multiple characters in a string?

ID : 10263

viewed : 23

Tags : pythonstringreplacepython

Top 5 Answer for python - Best way to replace multiple characters in a string?

vote vote

98

Replacing two characters

I timed all the methods in the current answers along with one extra.

With an input string of abc&def#ghi and replacing & -> \& and # -> \#, the fastest way was to chain together the replacements like this: text.replace('&', '\&').replace('#', '\#').

Timings for each function:

  • a) 1000000 loops, best of 3: 1.47 μs per loop
  • b) 1000000 loops, best of 3: 1.51 μs per loop
  • c) 100000 loops, best of 3: 12.3 μs per loop
  • d) 100000 loops, best of 3: 12 μs per loop
  • e) 100000 loops, best of 3: 3.27 μs per loop
  • f) 1000000 loops, best of 3: 0.817 μs per loop
  • g) 100000 loops, best of 3: 3.64 μs per loop
  • h) 1000000 loops, best of 3: 0.927 μs per loop
  • i) 1000000 loops, best of 3: 0.814 μs per loop

Here are the functions:

def a(text):     chars = "&#"     for c in chars:         text = text.replace(c, "\\" + c)   def b(text):     for ch in ['&','#']:         if ch in text:             text = text.replace(ch,"\\"+ch)   import re def c(text):     rx = re.compile('([&#])')     text = rx.sub(r'\\\1', text)   RX = re.compile('([&#])') def d(text):     text = RX.sub(r'\\\1', text)   def mk_esc(esc_chars):     return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s]) esc = mk_esc('&#') def e(text):     esc(text)   def f(text):     text = text.replace('&', '\&').replace('#', '\#')   def g(text):     replacements = {"&": "\&", "#": "\#"}     text = "".join([replacements.get(c, c) for c in text])   def h(text):     text = text.replace('&', r'\&')     text = text.replace('#', r'\#')   def i(text):     text = text.replace('&', r'\&').replace('#', r'\#') 

Timed like this:

python -mtimeit -s"import time_functions" "time_functions.a('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.b('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.c('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.d('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.e('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.f('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.g('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.h('abc&def#ghi')" python -mtimeit -s"import time_functions" "time_functions.i('abc&def#ghi')" 

Replacing 17 characters

Here's similar code to do the same but with more characters to escape (\`*_{}>#+-.!$):

def a(text):     chars = "\\`*_{}[]()>#+-.!$"     for c in chars:         text = text.replace(c, "\\" + c)   def b(text):     for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:         if ch in text:             text = text.replace(ch,"\\"+ch)   import re def c(text):     rx = re.compile('([&#])')     text = rx.sub(r'\\\1', text)   RX = re.compile('([\\`*_{}[]()>#+-.!$])') def d(text):     text = RX.sub(r'\\\1', text)   def mk_esc(esc_chars):     return lambda s: ''.join(['\\' + c if c in esc_chars else c for c in s]) esc = mk_esc('\\`*_{}[]()>#+-.!$') def e(text):     esc(text)   def f(text):     text = text.replace('\\', '\\\\').replace('`', '\`').replace('*', '\*').replace('_', '\_').replace('{', '\{').replace('}', '\}').replace('[', '\[').replace(']', '\]').replace('(', '\(').replace(')', '\)').replace('>', '\>').replace('#', '\#').replace('+', '\+').replace('-', '\-').replace('.', '\.').replace('!', '\!').replace('$', '\$')   def g(text):     replacements = {         "\\": "\\\\",         "`": "\`",         "*": "\*",         "_": "\_",         "{": "\{",         "}": "\}",         "[": "\[",         "]": "\]",         "(": "\(",         ")": "\)",         ">": "\>",         "#": "\#",         "+": "\+",         "-": "\-",         ".": "\.",         "!": "\!",         "$": "\$",     }     text = "".join([replacements.get(c, c) for c in text])   def h(text):     text = text.replace('\\', r'\\')     text = text.replace('`', r'\`')     text = text.replace('*', r'\*')     text = text.replace('_', r'\_')     text = text.replace('{', r'\{')     text = text.replace('}', r'\}')     text = text.replace('[', r'\[')     text = text.replace(']', r'\]')     text = text.replace('(', r'\(')     text = text.replace(')', r'\)')     text = text.replace('>', r'\>')     text = text.replace('#', r'\#')     text = text.replace('+', r'\+')     text = text.replace('-', r'\-')     text = text.replace('.', r'\.')     text = text.replace('!', r'\!')     text = text.replace('$', r'\$')   def i(text):     text = text.replace('\\', r'\\').replace('`', r'\`').replace('*', r'\*').replace('_', r'\_').replace('{', r'\{').replace('}', r'\}').replace('[', r'\[').replace(']', r'\]').replace('(', r'\(').replace(')', r'\)').replace('>', r'\>').replace('#', r'\#').replace('+', r'\+').replace('-', r'\-').replace('.', r'\.').replace('!', r'\!').replace('$', r'\$') 

Here's the results for the same input string abc&def#ghi:

  • a) 100000 loops, best of 3: 6.72 μs per loop
  • b) 100000 loops, best of 3: 2.64 μs per loop
  • c) 100000 loops, best of 3: 11.9 μs per loop
  • d) 100000 loops, best of 3: 4.92 μs per loop
  • e) 100000 loops, best of 3: 2.96 μs per loop
  • f) 100000 loops, best of 3: 4.29 μs per loop
  • g) 100000 loops, best of 3: 4.68 μs per loop
  • h) 100000 loops, best of 3: 4.73 μs per loop
  • i) 100000 loops, best of 3: 4.24 μs per loop

And with a longer input string (## *Something* and [another] thing in a longer sentence with {more} things to replace$):

  • a) 100000 loops, best of 3: 7.59 μs per loop
  • b) 100000 loops, best of 3: 6.54 μs per loop
  • c) 100000 loops, best of 3: 16.9 μs per loop
  • d) 100000 loops, best of 3: 7.29 μs per loop
  • e) 100000 loops, best of 3: 12.2 μs per loop
  • f) 100000 loops, best of 3: 5.38 μs per loop
  • g) 10000 loops, best of 3: 21.7 μs per loop
  • h) 100000 loops, best of 3: 5.7 μs per loop
  • i) 100000 loops, best of 3: 5.13 μs per loop

Adding a couple of variants:

def ab(text):     for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:         text = text.replace(ch,"\\"+ch)   def ba(text):     chars = "\\`*_{}[]()>#+-.!$"     for c in chars:         if c in text:             text = text.replace(c, "\\" + c) 

With the shorter input:

  • ab) 100000 loops, best of 3: 7.05 μs per loop
  • ba) 100000 loops, best of 3: 2.4 μs per loop

With the longer input:

  • ab) 100000 loops, best of 3: 7.71 μs per loop
  • ba) 100000 loops, best of 3: 6.08 μs per loop

So I'm going to use ba for readability and speed.

Addendum

Prompted by haccks in the comments, one difference between ab and ba is the if c in text: check. Let's test them against two more variants:

def ab_with_check(text):     for ch in ['\\','`','*','_','{','}','[',']','(',')','>','#','+','-','.','!','$','\'']:         if ch in text:             text = text.replace(ch,"\\"+ch)  def ba_without_check(text):     chars = "\\`*_{}[]()>#+-.!$"     for c in chars:         text = text.replace(c, "\\" + c) 

Times in μs per loop on Python 2.7.14 and 3.6.3, and on a different machine from the earlier set, so cannot be compared directly.

╭────────────╥──────┬───────────────┬──────┬──────────────────╮ │ Py, input  ║  ab  │ ab_with_check │  ba  │ ba_without_check │ ╞════════════╬══════╪═══════════════╪══════╪══════════════════╡ │ Py2, short ║ 8.81 │    4.22       │ 3.45 │    8.01          │ │ Py3, short ║ 5.54 │    1.34       │ 1.46 │    5.34          │ ├────────────╫──────┼───────────────┼──────┼──────────────────┤ │ Py2, long  ║ 9.3  │    7.15       │ 6.85 │    8.55          │ │ Py3, long  ║ 7.43 │    4.38       │ 4.41 │    7.02          │ └────────────╨──────┴───────────────┴──────┴──────────────────┘ 

We can conclude that:

  • Those with the check are up to 4x faster than those without the check

  • ab_with_check is slightly in the lead on Python 3, but ba (with check) has a greater lead on Python 2

  • However, the biggest lesson here is Python 3 is up to 3x faster than Python 2! There's not a huge difference between the slowest on Python 3 and fastest on Python 2!

vote vote

80

>>> string="abc&def#ghi" >>> for ch in ['&','#']: ...   if ch in string: ...      string=string.replace(ch,"\\"+ch) ... >>> print string abc\&def\#ghi 
vote vote

79

Here is a python3 method using str.translate and str.maketrans:

s = "abc&def#ghi" print(s.translate(str.maketrans({'&': '\&', '#': '\#'}))) 

The printed string is abc\&def\#ghi.

vote vote

65

Simply chain the replace functions like this

strs = "abc&def#ghi" print strs.replace('&', '\&').replace('#', '\#') # abc\&def\#ghi 

If the replacements are going to be more in number, you can do this in this generic way

strs, replacements = "abc&def#ghi", {"&": "\&", "#": "\#"} print "".join([replacements.get(c, c) for c in strs]) # abc\&def\#ghi 
vote vote

57

Are you always going to prepend a backslash? If so, try

import re rx = re.compile('([&#])') #                  ^^ fill in the characters here. strs = rx.sub('\\\\\\1', strs) 

It may not be the most efficient method but I think it is the easiest.

Top 3 video Explaining python - Best way to replace multiple characters in a string?

Related QUESTION?