我正在編寫一個語法來解析HTSQL語法,并堅持如何處理段和除法運算符的/字符重用.

described grammar并不是非常正式,所以我一直在關注Python實現的確切輸出,從粗略的一瞥似乎是一個手寫的解析器,而不是使用解析器生成器 – 作為參考解析器生成器我目前使用是

CL-YACC和

CL-LEX.(FWIW完整的東西

is here,雖然可能有點過時.)

由于“/ 1”被解析為'(：COLLECT(：INTEGER“1”))’,但是“/ 1/2”被解析為'(：COLLECT(：OPERATOR /) (：INTEGER“1”)(：INTEGER“2”)))’,即一個是段分隔符,另一個是分區; “/ 1 // 2”再次是'(：COLLECT(：OPERATOR /(：INTEGER“1”)(：COLLECT(：INTEGER“2”))))’.

問題是,如何在不借助切換到手動解析器的情況下在語法規范中處理這個問題？切換到不同的解析器生成器類(而不是LALR(1))會有幫助嗎？

到目前為止,我已經嘗試了不同的語法變體,但是對于整個語法來說,優先級是固定的這一事實也會干擾斜杠的兩種解釋.我嘗試的另一種方法是在詞法分析器中消除歧義,即以不同方式處理第一個斜杠(在每個“組”中)并返回不同的符號,例如, DIV – 然而,我找不到一個好的規則,并以某種方式懷疑它純粹是通過查看詞匯結構而存在.

最后,我很想通過完全脫離給定的解析器來解決這個問題,以使我的生活更輕松.從某種意義上說,更容易理解可預測的語法,你會認為這更令人滿意嗎？

圖1,用于檢查解析樹的Python腳本：

import htsql

application = htsql.HTSQL("sqlite:///htsql_demo.sqlite")

global y

y = None

def p(string):

global y

with application:

y = htsql.core.syn.parse.parse(string)

return y

def l(name):

result = []

for c in name:

if c.isupper() and result:

result.append("-")

result.append(c)

return "".join(result)

def keyword(name):

return ":{}".format(name.upper())

def n(expression):

name = expression.__class__.__name__

name = name[:name.find("Syntax")]

return keyword(l(name))

def t(expression):

arguments = [n(expression)]

d = expression.__dict__

if "identifier" in d:

arguments.append(t(expression.identifier))

if "text" in d:

arguments.append("\"{}\"".format(expression.text))

if "symbol" in d:

if not isinstance(expression, (htsql.core.syn.syntax.ProjectSyntax, htsql.core.syn.syntax.FilterSyntax, htsql.core.syn.syntax.CollectSyntax, htsql.core.syn.syntax.DetachSyntax)):

arguments.append(expression.symbol)

if "arm" in d:

arguments.append(t(expression.arm))

if "larm" in d:

arguments.append(t(expression.larm))

if "rarm" in d:

arguments.append(t(expression.rarm))

if "arms" in d:

arguments.extend(t(x) for x in expression.arms)

if "rarms" in d:

arguments.extend(t(x) for x in expression.rarms)

return "({})".format(" ".join(arguments))

# t(p("/school"))

# '(:COLLECT (:IDENTIFIER "school"))

# t(p("/'school'"))

# '(:COLLECT (:STRING "school"))

圖2,我當前的解析器,沒有正確處理這個問題：

(defpackage #:cl-htsql

(:use #:cl #:alexandria #:cl-lex #:yacc)

(:import-from #:arnesi #:with-collector))

(eval-when (:compile-toplevel :load-toplevel :execute)

(defun maybe-intern (name &optional (package NIL package-p))

"If NAME is a SYMBOL, return it, otherwise INTERN it."

(cond

((symbolp name)

name)

(package-p

(intern name package))

(T

(intern name))))

(defmacro define-lexer (name &body patterns)

"Shortcut for DEFINE-STRING-LEXER."

`(define-string-lexer ,name

,@(mapcar

(lambda (pattern)

(etypecase pattern

((or symbol string)

(let ((symbol (maybe-intern pattern))

(pattern (string pattern)))

`(,pattern (return (values ',symbol ',symbol)))))

(list

(destructuring-bind (pattern &optional symbol value) pattern

(let* ((symbol (or symbol (intern pattern)))

(value (or value symbol)))

(etypecase symbol

(list

`(,pattern ,symbol))

(symbol

`(,pattern (return (values ',symbol ',value))))))))))

patterns))))

;; parser are results are to be treated immutable

(define-lexer string-lexer

/

("\\|" \|)

("\\&" &)

<=

>=

==

=

!==

!=

!~

!

~

<

>

@

("\\?" ?)

("\\." \.)

("\\(" \()

("\\)" \))

("\\+" +)

-

("\\*" *)

\:

("-?0|[1-9][0-9]*(\\.[0-9]*)?([eE][+-]?[0-9]+)?"

(return (cond

((find #\e $@)

(values 'float $@))

((find #\. $@)

(values 'decimal $@))

(T

(values 'integer $@)))))

("([^\"\\.\\?~\'=<>\\(\\)@\\|\\&/:])+" (return (values 'name $@)))

("\'([^\\\']|\\.)*?\'" (return (values 'string (string-trim "\'" $@))))

("\"([^\\\"]|\\.)*?\"" (return (values 'string (string-trim "\"" $@)))))

(define-parser *expression-parser*

(:muffle-conflicts (44 0))

(:start-symbol query)

(:terminals (|\|| #+(or)div & ! |.| ? / = != !== !~ ~ < > == <= >= \( \) + - * @ name integer decimal float string))

(:precedence ((:left @) (:left ~) (:left |.|) (:left + -) (:left * div) (:left = != == !== ~ !~ < <= > >=) (:left !) (:left &) (:left |\||) (:left ?) (:left /)))

(query

segment)

(segment

(/ segment (lambda (x y) (declare (ignore x)) (if (eq y :skip) '(:skip) `(:collect ,y))))

skip

group)

(skip

((constantly :skip)))

(group

(\( segment \) (lambda (x y z) (declare (ignore x z)) `(:group ,y)))

sieve)

(sieve

(segment ? segment (lambda (x y z) (declare (ignore y)) `(:filter ,x ,z)))

or)

(or

(segment |\|| segment (lambda (x y z) `(:operator ,y ,x ,z)))

and)

(and

(segment & segment (lambda (x y z) `(:operator ,y ,x ,z)))

not)

(not

(! segment (lambda (x y) `(:prefix ,x ,y)))

comparison)

(comparison

(segment = segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment != segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment ~ segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment !~ segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment == segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment !== segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment < segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment <= segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment > segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment >= segment (lambda (x y z) `(:operator ,y ,x ,z)))

addition)

(addition

(segment + segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment - segment (lambda (x y z) `(:operator ,y ,x ,z)))

multiplication)

(multiplication

(segment * segment (lambda (x y z) `(:operator ,y ,x ,z)))

(segment / segment (lambda (x y z) (declare (ignore y)) `(:operator / ,x ,z)))

composition)

(composition

(segment |.| segment (lambda (x y z) (declare (ignore y)) `(:compose ,x ,z)))

attach)

(attach

(segment @ segment (lambda (x y z) (declare (ignore y)) `(:attach ,x ,z)))

detach)

(detach

(@ segment (lambda (x y) (declare (ignore x)) `(:detach ,y)))

term)

(term

(name (lambda (x) `(:identifier ,x)))

(string (lambda (x) `(:string ,x)))

(number (lambda (x) `(:integer ,x)))

(integer (lambda (x) `(:integer ,x)))

(decimal (lambda (x) `(:decimal ,x)))

(float (lambda (x) `(:float ,x)))))

(defun make-lexer-for-source (source)

"Make a lexer for the SOURCE, either a STRING or a STREAM."

(etypecase source

(string (string-lexer source))

(stream

(flet ((ignore (c)

(declare (ignore c))))

(stream-lexer #'read-line #'string-lexer #'ignore #'ignore)))))

(defun lex-source (source)

"Debug helper to lex a SOURCE into a list of tokens."

(let ((lexer (make-lexer-for-source source)))

(loop

for (x y) = (multiple-value-list (funcall lexer))

while x

collect (list x y))))

(define-condition htsql-parse-error (simple-error) ())

(defun translate-yacc-error (error)

(make-condition

'htsql-parse-error

:format-control "Couldn't parse HTSQL query: ~A."

:format-arguments (list error)))

(defun parse-htsql-query (source)

"Parse SOURCE into a syntax tree. The SOURCE may be either a STRING or

a STREAM."

(handler-case

(parse-with-lexer

(make-lexer-for-source source)

*expression-parser*)

(yacc-parse-error (error)

(error (translate-yacc-error error)))))

;; > (parse-htsql-query "/1/")

;; (:OPERATOR / (:COLLECT (:INTEGER "1")) :SKIP)

;; > (parse-htsql-query "/1/2")

;; (:OPERATOR / (:COLLECT (:INTEGER "1")) (:INTEGER "2"))

總結

以上是生活随笔為你收集整理的python中ht_python – 解析HTSQL时处理语法歧义的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。